Understanding the Problem Statement & Business Case

  • Artificial Intelligence (AI), Machine Learning (ML) and Deep Learning (DL) have been transforming finance and investing.
  • "Artificial intelligence is to trading what fire was to the cavemen"!
  • Electronic trades account for almost 45% of revenues in cash equities trading"
  • AI powered robo-advisers can perform real-time analysis on massive datasets and trade securities at an extremely faster rate compared to human traders.
  • AI-powered trading could potentially reduce risk and maximize returns.
  • In this project, I trained a ridge regression model and deep neural network model to predict future stock prices.
  • By accurately predicting stock prices, investors can maximize returns and know when to buy/sell securities.
  • The AI/ML model will be trained using historical stock price data along with the volume of transactions.
  • We will use a type of neural nets known as Long Short-Term Memory Networks (LSTM).
  • Disclaimer: Stock prices are volatile and are generally hard to predict. Invest at your own risk.

Import Datasets and Libraries

In [1]:
import pandas as pd
import plotly.express as px
from copy import copy
from scipy import stats
import matplotlib.pyplot as plt
import numpy as np
import plotly.figure_factory as ff
from sklearn.linear_model import LinearRegression
from sklearn.linear_model import Ridge
from sklearn.svm import SVR
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score
from sklearn.preprocessing import MinMaxScaler
from tensorflow import keras
In [2]:
# Read stock prices data
stock_price_df = pd.read_csv("D:\Python and Machine Learning for Financial Analysis\stock.csv")
stock_price_df
Out[2]:
Date AAPL BA T MGM AMZN IBM TSLA GOOG sp500
0 2012-01-12 60.198570 75.510002 30.120001 12.130000 175.929993 180.550003 28.250000 313.644379 1295.500000
1 2012-01-13 59.972858 74.599998 30.070000 12.350000 178.419998 179.160004 22.790001 311.328064 1289.089966
2 2012-01-17 60.671429 75.239998 30.250000 12.250000 181.660004 180.000000 26.600000 313.116364 1293.670044
3 2012-01-18 61.301430 75.059998 30.330000 12.730000 189.440002 181.070007 26.809999 315.273285 1308.040039
4 2012-01-19 61.107143 75.559998 30.420000 12.800000 194.449997 180.520004 26.760000 318.590851 1314.500000
... ... ... ... ... ... ... ... ... ... ...
2154 2020-08-05 440.250000 174.279999 29.850000 16.719999 3205.030029 125.449997 1485.020020 1473.609985 3327.770020
2155 2020-08-06 455.609985 172.199997 29.840000 18.459999 3225.000000 126.120003 1489.579956 1500.099976 3349.159912
2156 2020-08-07 444.450012 170.020004 30.020000 19.030001 3167.459961 124.959999 1452.709961 1494.489990 3351.280029
2157 2020-08-10 450.910004 179.410004 30.200001 21.650000 3148.159912 127.110001 1418.569946 1496.099976 3360.469971
2158 2020-08-11 437.500000 180.130005 30.200001 21.500000 3080.669922 126.750000 1374.390015 1480.319946 3333.689941

2159 rows × 10 columns

In [3]:
# Read the stocks volume data
stock_vol_df = pd.read_csv("D:\Python and Machine Learning for Financial Analysis\stock_volume.csv")
stock_vol_df
Out[3]:
Date AAPL BA T MGM AMZN IBM TSLA GOOG sp500
0 2012-01-12 53146800 3934500 26511100 17891100 5385800 6881000 729300 3764400 4019890000
1 2012-01-13 56505400 4641100 22096800 16621800 4753500 5279200 5500400 4631800 3692370000
2 2012-01-17 60724300 3700100 23500200 15480800 5644500 6003400 4651600 3832800 4010490000
3 2012-01-18 69197800 4189500 22015000 18387600 7473500 4600600 1260200 5544000 4096160000
4 2012-01-19 65434600 5397300 25524000 14022900 7096000 8567200 1246300 12657800 4465890000
... ... ... ... ... ... ... ... ... ... ...
2154 2020-08-05 30498000 46551000 22991700 18914200 3930000 3675400 4978000 1979500 4732220000
2155 2020-08-06 50607200 32921600 21908700 35867700 3940600 3417100 5992300 1995400 4267490000
2156 2020-08-07 49453300 19301600 30398500 34530300 3929600 3651000 8883500 1576600 4104860000
2157 2020-08-10 53100900 35857700 35514400 71219700 3167300 3968300 7522300 1289300 4318570000
2158 2020-08-11 46871100 60966900 30978300 34357900 3706600 4998500 8356000 1452000 5087650000

2159 rows × 10 columns

In [4]:
# Sorted the data based on Date
stock_price_df = stock_price_df.sort_values(by = ['Date'])
stock_price_df
Out[4]:
Date AAPL BA T MGM AMZN IBM TSLA GOOG sp500
0 2012-01-12 60.198570 75.510002 30.120001 12.130000 175.929993 180.550003 28.250000 313.644379 1295.500000
1 2012-01-13 59.972858 74.599998 30.070000 12.350000 178.419998 179.160004 22.790001 311.328064 1289.089966
2 2012-01-17 60.671429 75.239998 30.250000 12.250000 181.660004 180.000000 26.600000 313.116364 1293.670044
3 2012-01-18 61.301430 75.059998 30.330000 12.730000 189.440002 181.070007 26.809999 315.273285 1308.040039
4 2012-01-19 61.107143 75.559998 30.420000 12.800000 194.449997 180.520004 26.760000 318.590851 1314.500000
... ... ... ... ... ... ... ... ... ... ...
2154 2020-08-05 440.250000 174.279999 29.850000 16.719999 3205.030029 125.449997 1485.020020 1473.609985 3327.770020
2155 2020-08-06 455.609985 172.199997 29.840000 18.459999 3225.000000 126.120003 1489.579956 1500.099976 3349.159912
2156 2020-08-07 444.450012 170.020004 30.020000 19.030001 3167.459961 124.959999 1452.709961 1494.489990 3351.280029
2157 2020-08-10 450.910004 179.410004 30.200001 21.650000 3148.159912 127.110001 1418.569946 1496.099976 3360.469971
2158 2020-08-11 437.500000 180.130005 30.200001 21.500000 3080.669922 126.750000 1374.390015 1480.319946 3333.689941

2159 rows × 10 columns

In [5]:
# Sorted the data based on Date
stock_vol_df = stock_vol_df.sort_values(by = ['Date'])
stock_vol_df
Out[5]:
Date AAPL BA T MGM AMZN IBM TSLA GOOG sp500
0 2012-01-12 53146800 3934500 26511100 17891100 5385800 6881000 729300 3764400 4019890000
1 2012-01-13 56505400 4641100 22096800 16621800 4753500 5279200 5500400 4631800 3692370000
2 2012-01-17 60724300 3700100 23500200 15480800 5644500 6003400 4651600 3832800 4010490000
3 2012-01-18 69197800 4189500 22015000 18387600 7473500 4600600 1260200 5544000 4096160000
4 2012-01-19 65434600 5397300 25524000 14022900 7096000 8567200 1246300 12657800 4465890000
... ... ... ... ... ... ... ... ... ... ...
2154 2020-08-05 30498000 46551000 22991700 18914200 3930000 3675400 4978000 1979500 4732220000
2155 2020-08-06 50607200 32921600 21908700 35867700 3940600 3417100 5992300 1995400 4267490000
2156 2020-08-07 49453300 19301600 30398500 34530300 3929600 3651000 8883500 1576600 4104860000
2157 2020-08-10 53100900 35857700 35514400 71219700 3167300 3968300 7522300 1289300 4318570000
2158 2020-08-11 46871100 60966900 30978300 34357900 3706600 4998500 8356000 1452000 5087650000

2159 rows × 10 columns

In [6]:
# Checked if Null values exist in stock prices data
stock_price_df.isnull().sum()
Out[6]:
Date     0
AAPL     0
BA       0
T        0
MGM      0
AMZN     0
IBM      0
TSLA     0
GOOG     0
sp500    0
dtype: int64
In [7]:
# Checked if Null values exist in stocks volume data
stock_vol_df.isnull().sum()
Out[7]:
Date     0
AAPL     0
BA       0
T        0
MGM      0
AMZN     0
IBM      0
TSLA     0
GOOG     0
sp500    0
dtype: int64
In [8]:
# Got stock prices dataframe info
stock_price_df.info()
<class 'pandas.core.frame.DataFrame'>
Int64Index: 2159 entries, 0 to 2158
Data columns (total 10 columns):
 #   Column  Non-Null Count  Dtype  
---  ------  --------------  -----  
 0   Date    2159 non-null   object 
 1   AAPL    2159 non-null   float64
 2   BA      2159 non-null   float64
 3   T       2159 non-null   float64
 4   MGM     2159 non-null   float64
 5   AMZN    2159 non-null   float64
 6   IBM     2159 non-null   float64
 7   TSLA    2159 non-null   float64
 8   GOOG    2159 non-null   float64
 9   sp500   2159 non-null   float64
dtypes: float64(9), object(1)
memory usage: 185.5+ KB
In [9]:
# Got stock volume dataframe info
stock_vol_df.info()
<class 'pandas.core.frame.DataFrame'>
Int64Index: 2159 entries, 0 to 2158
Data columns (total 10 columns):
 #   Column  Non-Null Count  Dtype 
---  ------  --------------  ----- 
 0   Date    2159 non-null   object
 1   AAPL    2159 non-null   int64 
 2   BA      2159 non-null   int64 
 3   T       2159 non-null   int64 
 4   MGM     2159 non-null   int64 
 5   AMZN    2159 non-null   int64 
 6   IBM     2159 non-null   int64 
 7   TSLA    2159 non-null   int64 
 8   GOOG    2159 non-null   int64 
 9   sp500   2159 non-null   int64 
dtypes: int64(9), object(1)
memory usage: 185.5+ KB
In [10]:
# Got the statistical data for the stocks volume dataframe
stock_vol_df.describe()
Out[10]:
AAPL BA T MGM AMZN IBM TSLA GOOG sp500
count 2.159000e+03 2.159000e+03 2.159000e+03 2.159000e+03 2.159000e+03 2.159000e+03 2.159000e+03 2.159000e+03 2.159000e+03
mean 5.820332e+07 6.419916e+06 2.832131e+07 9.845582e+06 4.102673e+06 4.453090e+06 7.001302e+06 2.498238e+06 3.680732e+09
std 4.568141e+07 9.711873e+06 1.428911e+07 7.295753e+06 2.290722e+06 2.462811e+06 5.781208e+06 1.928407e+06 8.622717e+08
min 1.136200e+07 7.889000e+05 6.862400e+06 9.507000e+05 8.813000e+05 1.193000e+06 3.649000e+05 7.900000e+03 1.248960e+09
25% 2.769930e+07 3.031850e+06 2.002150e+07 5.796450e+06 2.675700e+06 3.111250e+06 3.433450e+06 1.325400e+06 3.211890e+09
50% 4.209420e+07 3.991000e+06 2.485930e+07 7.899800e+06 3.494800e+06 3.825000e+06 5.581100e+06 1.813900e+06 3.526890e+09
75% 7.182480e+07 5.325900e+06 3.210565e+07 1.104055e+07 4.768150e+06 4.937300e+06 8.619550e+06 3.245350e+06 3.933290e+09
max 3.765300e+08 1.032128e+08 1.950827e+08 9.009820e+07 2.385610e+07 3.049020e+07 6.093880e+07 2.497790e+07 9.044690e+09
  • Average trading volume for Apple stock is 5.820332e+07

  • Maximum trading volume for S&P500 is 9.044690e+09

  • S&P500 is the most traded security of them all.

  • The S&P 500 index is a broad-based measure of large corporations traded on U.S. stock markets. Over long periods of time, passively holding the index often produces better results than actively trading or picking single stocks.
  • Over long-time horizons, the index typically produces better returns than actively managed portfolios.
In [11]:
# Got the statistical data for the prices dataframe
stock_price_df.describe()
Out[11]:
AAPL BA T MGM AMZN IBM TSLA GOOG sp500
count 2159.000000 2159.000000 2159.000000 2159.000000 2159.000000 2159.000000 2159.000000 2159.000000 2159.000000
mean 140.819823 189.942700 35.162899 23.105743 915.665665 161.853001 259.600815 783.712512 2218.749554
std 70.827601 103.678586 3.207490 6.963847 697.838905 25.561938 210.988003 334.448057 537.321727
min 55.790001 67.239998 26.770000 7.140000 175.929993 94.769997 22.790001 278.481171 1278.040039
25% 89.165714 124.015000 33.040001 18.545000 316.490005 142.769997 184.595001 527.214416 1847.984985
50% 116.599998 142.419998 34.930000 23.780001 676.010010 156.949997 231.960007 737.599976 2106.629883
75% 175.019997 297.044998 37.419998 28.430000 1593.645019 185.974998 307.350006 1079.744995 2705.810059
max 455.609985 440.619995 43.470001 38.029999 3225.000000 215.800003 1643.000000 1568.489990 3386.149902
  • Average Stock Price for S&P500 = 2218.749554
  • Maximum Tesla Stock Price = 1643.000000

Performed Exploratory Data Analysis and Visualization

In [12]:
# Function to normalize stock prices based on their initial price
def normalize(df):
  x = df.copy()
  for i in x.columns[1:]:
    x[i] = x[i]/x[i][0]
  return x
In [13]:
# Function to plot interactive plots using Plotly Express
def interactive_plot(df, title):
  fig = px.line(title = title)
  for i in df.columns[1:]:
    fig.add_scatter(x = df['Date'], y = df[i], name = i)
  fig.show()
In [14]:
# Plotted interactive chart for stocks data
interactive_plot(stock_price_df, 'Stock Prices')
In [15]:
# Plotted interactive chart for volume data
# Noticed that S&P500 trading is orders of magnitude compared to individual stocks
interactive_plot(stock_vol_df, 'Stocks Volume')
In [16]:
# Plotted interactive chart for normalized stocks prices data
interactive_plot(normalize(stock_price_df), 'Stock Prices')

# Let's normalize the data and re-plot interactive chart for volume data
interactive_plot(normalize(stock_vol_df), 'Normalized Volume')

Prepared the Data before Training the AI/ML Model

Training & Testing Data Split

  • Data set is divided into 75% for training and 25% for testing.
    • Training set: used for model training.
    • Testing set: used for testing trained model. Made sure that the testing dataset has never been seen by the trained model before.
In [17]:
# Function to concatenate the date, stock price, and volume in one dataframe
def individual_stock(price_df, vol_df, name):
    return pd.DataFrame({'Date': price_df['Date'], 'Close': price_df[name], 'Volume': vol_df[name]})
In [18]:
# Function to return the input/output (target) data for AI/ML Model
# Noted that our goal is to predict the future stock price 
# Target stock price today will be tomorrow's price 
def trading_window(data):
  
  # 1 day window 
  n = 1

  # Created a column containing the prices for the next 1 days
  data['Target'] = data[['Close']].shift(-n)
  
  # Returned the new dataset 
  return data
In [19]:
# Let's test the functions and get individual stock prices and volumes for AAPL
price_volume_df = individual_stock(stock_price_df, stock_vol_df, 'AAPL')
price_volume_df
Out[19]:
Date Close Volume
0 2012-01-12 60.198570 53146800
1 2012-01-13 59.972858 56505400
2 2012-01-17 60.671429 60724300
3 2012-01-18 61.301430 69197800
4 2012-01-19 61.107143 65434600
... ... ... ...
2154 2020-08-05 440.250000 30498000
2155 2020-08-06 455.609985 50607200
2156 2020-08-07 444.450012 49453300
2157 2020-08-10 450.910004 53100900
2158 2020-08-11 437.500000 46871100

2159 rows × 3 columns

In [20]:
price_volume_target_df = trading_window(price_volume_df)
price_volume_target_df
Out[20]:
Date Close Volume Target
0 2012-01-12 60.198570 53146800 59.972858
1 2012-01-13 59.972858 56505400 60.671429
2 2012-01-17 60.671429 60724300 61.301430
3 2012-01-18 61.301430 69197800 61.107143
4 2012-01-19 61.107143 65434600 60.042858
... ... ... ... ...
2154 2020-08-05 440.250000 30498000 455.609985
2155 2020-08-06 455.609985 50607200 444.450012
2156 2020-08-07 444.450012 49453300 450.910004
2157 2020-08-10 450.910004 53100900 437.500000
2158 2020-08-11 437.500000 46871100 NaN

2159 rows × 4 columns

In [21]:
# Removed the last row as it will be a null value
price_volume_target_df = price_volume_target_df[:-1]
price_volume_target_df
Out[21]:
Date Close Volume Target
0 2012-01-12 60.198570 53146800 59.972858
1 2012-01-13 59.972858 56505400 60.671429
2 2012-01-17 60.671429 60724300 61.301430
3 2012-01-18 61.301430 69197800 61.107143
4 2012-01-19 61.107143 65434600 60.042858
... ... ... ... ...
2153 2020-08-04 438.660004 43267900 440.250000
2154 2020-08-05 440.250000 30498000 455.609985
2155 2020-08-06 455.609985 50607200 444.450012
2156 2020-08-07 444.450012 49453300 450.910004
2157 2020-08-10 450.910004 53100900 437.500000

2158 rows × 4 columns

In [22]:
# Scaled the data
sc = MinMaxScaler(feature_range = (0, 1))
price_volume_target_scaled_df = sc.fit_transform(price_volume_target_df.drop(columns = ['Date']))
In [23]:
price_volume_target_scaled_df
Out[23]:
array([[0.01102638, 0.11442624, 0.01046185],
       [0.01046185, 0.12362365, 0.01220906],
       [0.01220906, 0.13517696, 0.01378478],
       ...,
       [1.        , 0.10747163, 0.97208751],
       [0.97208751, 0.10431171, 0.98824476],
       [0.98824476, 0.11430054, 0.95470465]])
In [24]:
price_volume_target_scaled_df.shape
Out[24]:
(2158, 3)
In [25]:
# Created Feature and Target
X = price_volume_target_scaled_df[:,:2]
y = price_volume_target_scaled_df[:,2:]
In [26]:
# Converted dataframe to arrays
# X = np.asarray(X)
# y = np.asarray(y)
X.shape, y.shape
Out[26]:
((2158, 2), (2158, 1))
In [27]:
# Split the data this way, since order is important in time-series
# Noted that we did not use train test split with it's default settings since it shuffles the data
split = int(0.65 * len(X))
X_train = X[:split]
y_train = y[:split]
X_test = X[split:]
y_test = y[split:]
In [28]:
X_train.shape, y_train.shape
Out[28]:
((1402, 2), (1402, 1))
In [29]:
X_test.shape, y_test.shape
Out[29]:
((756, 2), (756, 1))
In [30]:
# Defined a data plotting function
def show_plot(data, title):
  plt.figure(figsize = (13, 5))
  plt.plot(data, linewidth = 3)
  plt.title(title)
  plt.grid()

show_plot(X_train, 'Training Data')
show_plot(X_test, 'Testing Data')
In [31]:
# Let's test the functions and get individual stock prices and volumes for S&P500
price_volume_df = individual_stock(stock_price_df, stock_vol_df, 'sp500')
price_volume_df
Out[31]:
Date Close Volume
0 2012-01-12 1295.500000 4019890000
1 2012-01-13 1289.089966 3692370000
2 2012-01-17 1293.670044 4010490000
3 2012-01-18 1308.040039 4096160000
4 2012-01-19 1314.500000 4465890000
... ... ... ...
2154 2020-08-05 3327.770020 4732220000
2155 2020-08-06 3349.159912 4267490000
2156 2020-08-07 3351.280029 4104860000
2157 2020-08-10 3360.469971 4318570000
2158 2020-08-11 3333.689941 5087650000

2159 rows × 3 columns

In [32]:
# Let's test the functions and get individual stock prices and volumes for Amazon 
price_volume_df = individual_stock(stock_price_df, stock_vol_df, 'AMZN')
price_volume_df
Out[32]:
Date Close Volume
0 2012-01-12 175.929993 5385800
1 2012-01-13 178.419998 4753500
2 2012-01-17 181.660004 5644500
3 2012-01-18 189.440002 7473500
4 2012-01-19 194.449997 7096000
... ... ... ...
2154 2020-08-05 3205.030029 3930000
2155 2020-08-06 3225.000000 3940600
2156 2020-08-07 3167.459961 3929600
2157 2020-08-10 3148.159912 3167300
2158 2020-08-11 3080.669922 3706600

2159 rows × 3 columns

Understanding the Theory and Intuition behind Regression

Simple Linear Regression: Intuition

  • In simple linear regression, we predict the value of one variable Y based on another variable X.
  • X is called the independent variable and Y is called the dependant variable.
  • It is called simple because it examines relationship between two variables only.
  • It is called linear as when the independent variable increases (or decreases), the dependent variable increases (or decreases) in a linear fashion.

M and B

  • Once the coefficients m and b are obtained, you have obtained a simple linear regression model!
  • This "trained" model can be later used to predict.

Simple Linear Regression: To Obtain Model Parameters using Least Sum of Squares

  • Least squares fitting is a way to find the best fit curve or line for a set of points.
  • The sum of the squares of the offsets (residuals) are used to estimate the best fit curve or line.
  • Least squares method is used to obtain the coefficients m and b.

Understanding the Concept of Regularization & Ridge Regression

Regularization: Intuition

  • Regularization techniques are used to avoid networks overfitting
  • Overfitting occurs when the model provide great results on the training data but performs poorly on testing dataset.
  • Overfitting occurs when the model learns all the patterns of the training dataset but fails to generalize.
  • Overfitted models generally provide high accuracy on training dataset but low accuracy on testing and validation (evaluation) datasets

Ridge Regression (L2 Regularization): Intuition

  • Ridge regression advantage is to avoid overfitting.
  • Our ultimate model is the one that could generalize patterns; i.e.: works best on the training and testing dataset
  • Overfitting occurs when the trained model performs well on the training data and performs poorly on the testing datasets
  • Ridge regression works by applying a penalizing term (reducing the weights and biases) to overcome overfitting.
  • Ridge regression works by attempting at increasing the bias to improve variance (generalization capability)
  • This works by changing the slope of the line
  • The model performance might become little poor on the training set but it will perform consistently well on both the training and testing datasets.

Ridge Regression (L2 Regularization): Math

  • Slope when reduced with the ridge regression penalty makes the model become less sensitive to changes in the independent variable

Ridge Regression (L2 Regularization): Alpha Effect

  • As Alpha increases, the slope of the regression line reduces and becomes more horizontal.
  • As Alpha increases, the model becomes less sensitive to the variations of the independent variable

Built and Trained a Ridge Linear Regression Model

In [33]:
# Noted that Ridge regression performs linear least squares with L2 regularization.
# Created and trained the Ridge Linear Regression  Model
regression_model = Ridge()
regression_model.fit(X_train, y_train)
Out[33]:
Ridge(alpha=1.0, copy_X=True, fit_intercept=True, max_iter=None,
      normalize=False, random_state=None, solver='auto', tol=0.001)
In [34]:
# Tested the model and calculated its accuracy 
lr_accuracy = regression_model.score(X_test, y_test)
print("Linear Regression Score: ", lr_accuracy)
Linear Regression Score:  0.7950028030821767
In [35]:
# Made Prediction
predicted_prices = regression_model.predict(X)
predicted_prices
Out[35]:
array([[0.03466412],
       [0.03374627],
       [0.03451936],
       ...,
       [0.81048342],
       [0.78876033],
       [0.80091324]])
In [36]:
# Append the predicted values into a list
Predicted = []
for i in predicted_prices:
  Predicted.append(i[0])
In [37]:
len(Predicted)
Out[37]:
2158
In [38]:
# Append the close values to the list
close = []
for i in price_volume_target_scaled_df:
  close.append(i[0])
In [39]:
# Created a dataframe based on the dates in the individual stock data
df_predicted = price_volume_target_df[['Date']]
df_predicted
Out[39]:
Date
0 2012-01-12
1 2012-01-13
2 2012-01-17
3 2012-01-18
4 2012-01-19
... ...
2153 2020-08-04
2154 2020-08-05
2155 2020-08-06
2156 2020-08-07
2157 2020-08-10

2158 rows × 1 columns

In [40]:
# Added the close values to the dataframe
df_predicted['Close'] = close
df_predicted
Out[40]:
Date Close
0 2012-01-12 0.011026
1 2012-01-13 0.010462
2 2012-01-17 0.012209
3 2012-01-18 0.013785
4 2012-01-19 0.013299
... ... ...
2153 2020-08-04 0.957606
2154 2020-08-05 0.961583
2155 2020-08-06 1.000000
2156 2020-08-07 0.972088
2157 2020-08-10 0.988245

2158 rows × 2 columns

In [41]:
# Added the predicted values to the dataframe
df_predicted['Prediction'] = Predicted
df_predicted
Out[41]:
Date Close Prediction
0 2012-01-12 0.011026 0.034664
1 2012-01-13 0.010462 0.033746
2 2012-01-17 0.012209 0.034519
3 2012-01-18 0.013785 0.034556
4 2012-01-19 0.013299 0.034707
... ... ... ...
2153 2020-08-04 0.957606 0.778280
2154 2020-08-05 0.961583 0.783205
2155 2020-08-06 1.000000 0.810483
2156 2020-08-07 0.972088 0.788760
2157 2020-08-10 0.988245 0.800913

2158 rows × 3 columns

In [42]:
# Plotted the results
interactive_plot(df_predicted, "Original Vs. Prediction")
In [43]:
# Noted that Ridge regression performs linear least squares with L2 regularization.
# Created and trained the Ridge Linear Regression  Model
regression_model_2 = Ridge(alpha = 2)
regression_model_2.fit(X_train, y_train)

# Tested the model and calculated its accuracy 
lr_accuracy = regression_model_2.score(X_test, y_test)
print("Linear Regression Score: ", lr_accuracy)

# Made Prediction
predicted_prices = regression_model_2.predict(X)
predicted_prices

# Append the predicted values into a list
Predicted = []
for i in predicted_prices:
  Predicted.append(i[0])

# Append the close values to the list
close = []
for i in price_volume_target_scaled_df:
  close.append(i[0])

# Created a dataframe based on the dates in the individual stock data
df_predicted_2 = price_volume_target_df[['Date']]
df_predicted_2

# Added the close values to the dataframe
df_predicted_2['Close'] = close
df_predicted_2

# Added the predicted values to the dataframe
df_predicted_2['Prediction'] = Predicted
df_predicted_2

# Plotted the results
interactive_plot(df_predicted_2, "Original Vs. Prediction")
Linear Regression Score:  0.4710364276441908

Understanding the Theory and Intuition behind Neural Networks

Neuron Mathematical Model

  • Artificial Neural Networks are information processing models that are inspired by the human brain.
  • The neuron collects signals from input channels named dendrites, processes information in its nucleus, and then generates an output in a long thin branch called axon.

Understanding how do Artificial Neural Networks Train

ANN Training using Gradient Descent

  • Gradient descent is an optimization algorithm used to obtain the optimized network weight and bias values.
  • It works by iteratively trying to minimize the cost function.
  • It works by calculating the gradient of the cost function and moving in the negative direction until the local/global minimum is achieved.
  • The size of the steps taken are called the learning rate
  • If learning rate increases, the area covered in the search space will increase so we might reach global minimum faster
  • However, we can overshoot the target
  • For small learning rates, training will take much longer to reach optimized weights values

Understanding the Theory and Intuition behind Recurrent Neural Networks

Recurrent Neural Networks (RNN):

  • Feedforward Neural Networks (vanilla networks) map a fixed size input (such as image) to a fixed size output (classes or probabilities).
  • A drawback in Feedforward networks is that they do not have any time dependency or memory effect.
  • A RNN is a type of ANN that is designed to take temporal dimension into consideration by having a memory (internal state) (feedback loop).

RNN Architecture

  • A RNN contains a temporal loop in which the hidden layer not only gives an output but it feeds itself as well.
  • An extra dimension is added which is time!
  • RNN can recall what happened in the previous time stamp so it works great with sequence of text.

RNNs are Special

  • Feedforward ANNs are so constrained with their fixed number of input and outputs.
  • For example, a CNN will have fixed size image and generates a fixed output (class or probabilities).
  • Feedforward ANN have a fixed configuration, i.e.: same number of hidden layers and weights.
  • Recurrent Neural Networks offer huge advantage over feedforward ANN and they are much more fun!
  • RNN allows us to work with a sequence of vectors:
    • Sequence in inputs
    • Sequence in outputs
    • Sequence in both!

RNN Math

  • A RNN accepts an input x and generate an output o.
  • The output o does not depend on the input x alone, however, it depends on the entire history of the inputs that have been fed to the network in previous time steps.

Understanding the Theory and Intuition behind Long Short Term Memory Networks

Vanishing Gradient Problem

  • LSTM networks work much better compared to vanilla RNN since they overcome the vanishing gradient problem.
  • The error has to propogate through all the previous layers resulting in a vanishing gradient.
  • As the gradient goes smaller, the network weights are no longer updated.
  • As more layers are added, the gradients of the loss function approaches zero, making the network hard to train.
  • ANN gradients are calculated during backpropagation.
  • In backpropagation, we calculate the derivatives of the network by moving from the outermost layer (close to output) back to the initial layers (close to inputs).
  • The chain rule is used during this calculation in which the derivatives from the final layers are multiplied by the derivaties from early layers.
  • The gradients keeps diminishing exponentially and therefore the weights and biases are no longer being updated.

LSTM Intuition

  • LSTM networks work better compared to vanilla RNN since they overcome vanishing gradient problem.
  • In practice, RNN fail to establish long term dependencies.
  • LSTM networks are type of RNN that are designed to remember long term dependencies by default.
  • LSTM can remember and recall information for a prolonged period of time.

LSTM Intuition - Gates

  • LSTM contains gates that can allow or block information passing by.
  • Gates consist of a sigmoid neural net layer along with a pointwise multiplication operation.
  • Sigmoid output ranges from 0 to 1:
    • 0 = Don't allow any data to flow
    • 1 = Allow everything to flow!

Trained an LSTM Time Series Model

In [44]:
# Let's test the functions and get individual stock prices and volumes for AAPL
price_volume_df = individual_stock(stock_price_df, stock_vol_df, 'AAPL')
price_volume_df
Out[44]:
Date Close Volume
0 2012-01-12 60.198570 53146800
1 2012-01-13 59.972858 56505400
2 2012-01-17 60.671429 60724300
3 2012-01-18 61.301430 69197800
4 2012-01-19 61.107143 65434600
... ... ... ...
2154 2020-08-05 440.250000 30498000
2155 2020-08-06 455.609985 50607200
2156 2020-08-07 444.450012 49453300
2157 2020-08-10 450.910004 53100900
2158 2020-08-11 437.500000 46871100

2159 rows × 3 columns

In [45]:
# Got the close and volume data as training data (Input)
training_data = price_volume_df.iloc[:, 1:3].values
training_data
Out[45]:
array([[6.01985700e+01, 5.31468000e+07],
       [5.99728580e+01, 5.65054000e+07],
       [6.06714290e+01, 6.07243000e+07],
       ...,
       [4.44450012e+02, 4.94533000e+07],
       [4.50910004e+02, 5.31009000e+07],
       [4.37500000e+02, 4.68711000e+07]])
In [46]:
# Normalized the data
sc = MinMaxScaler(feature_range = (0, 1))
training_set_scaled = sc.fit_transform(training_data)
In [47]:
# Created the training and testing data, training data contains present day and previous day values
X = []
y = []
for i in range(1, len(price_volume_df)):
    X.append(training_set_scaled [i-1:i, 0])
    y.append(training_set_scaled [i, 0])
In [48]:
X
Out[48]:
[array([0.01102638]),
 array([0.01046185]),
 array([0.01220906]),
 array([0.01378478]),
 array([0.01329884]),
 array([0.01063693]),
 array([0.01317736]),
 array([0.01067623]),
 array([0.02005545]),
 array([0.01933012]),
 array([0.02027698]),
 array([0.02232433]),
 array([0.02356417]),
 array([0.02346055]),
 array([0.02307824]),
 array([0.02470754]),
 array([0.02624037]),
 array([0.02797688]),
 array([0.03078171]),
 array([0.03667365]),
 array([0.03676297]),
 array([0.04004303]),
 array([0.04249412]),
 array([0.0382815]),
 array([0.03990367]),
 array([0.03987152]),
 array([0.04442]),
 array([0.04377327]),
 array([0.04497023]),
 array([0.0471212]),
 array([0.04831818]),
 array([0.05176615]),
 array([0.054278]),
 array([0.05500331]),
 array([0.05525701]),
 array([0.05096223]),
 array([0.04992603]),
 array([0.05007968]),
 array([0.0541172]),
 array([0.05525344]),
 array([0.05769381]),
 array([0.06344641]),
 array([0.07112129]),
 array([0.06968493]),
 array([0.0696885]),
 array([0.07523743]),
 array([0.07697392]),
 array([0.07573765]),
 array([0.07460858]),
 array([0.07343305]),
 array([0.07733837]),
 array([0.08001816]),
 array([0.08114009]),
 array([0.07836741]),
 array([0.07468361]),
 array([0.08150097]),
 array([0.08532053]),
 array([0.08353044]),
 array([0.08687837]),
 array([0.0877895]),
 array([0.08500612]),
 array([0.08420576]),
 array([0.08298019]),
 array([0.07671309]),
 array([0.06774478]),
 array([0.07831024]),
 array([0.07782431]),
 array([0.07035666]),
 array([0.06519005]),
 array([0.06473269]),
 array([0.0606523]),
 array([0.07841744]),
 array([0.07759563]),
 array([0.07591631]),
 array([0.06912039]),
 array([0.06845938]),
 array([0.06983499]),
 array([0.06834861]),
 array([0.06242809]),
 array([0.06393949]),
 array([0.06347499]),
 array([0.0638323]),
 array([0.06431107]),
 array([0.06294976]),
 array([0.05991624]),
 array([0.05811186]),
 array([0.05557858]),
 array([0.04987602]),
 array([0.04996891]),
 array([0.06100959]),
 array([0.05946963]),
 array([0.06432537]),
 array([0.06245311]),
 array([0.06137047]),
 array([0.06493635]),
 array([0.06740176]),
 array([0.06688725]),
 array([0.06090597]),
 array([0.06208507]),
 array([0.06156343]),
 array([0.06464696]),
 array([0.06473985]),
 array([0.06781265]),
 array([0.06454332]),
 array([0.06632627]),
 array([0.06489706]),
 array([0.06467195]),
 array([0.06560095]),
 array([0.06976353]),
 array([0.07034595]),
 array([0.06974923]),
 array([0.0668658]),
 array([0.06844866]),
 array([0.06440042]),
 array([0.06485062]),
 array([0.06573315]),
 array([0.06378584]),
 array([0.06912754]),
 array([0.07217176]),
 array([0.07463358]),
 array([0.07839599]),
 array([0.07694533]),
 array([0.07980734]),
 array([0.07777787]),
 array([0.07642726]),
 array([0.07445136]),
 array([0.07662019]),
 array([0.07731337]),
 array([0.07732407]),
 array([0.07708112]),
 array([0.07996099]),
 array([0.07638081]),
 array([0.07621287]),
 array([0.07517311]),
 array([0.06590109]),
 array([0.06586893]),
 array([0.069542]),
 array([0.0730686]),
 array([0.07868898]),
 array([0.07727763]),
 array([0.07762779]),
 array([0.08045407]),
 array([0.08290159]),
 array([0.08231562]),
 array([0.08194045]),
 array([0.0822513]),
 array([0.08259788]),
 array([0.0855635]),
 array([0.08616736]),
 array([0.08586006]),
 array([0.08782881]),
 array([0.09203427]),
 array([0.09812273]),
 array([0.09487483]),
 array([0.09945189]),
 array([0.09722233]),
 array([0.09743313]),
 array([0.10188513]),
 array([0.10157071]),
 array([0.10109549]),
 array([0.09766538]),
 array([0.09815489]),
 array([0.10163145]),
 array([0.09993784]),
 array([0.10209595]),
 array([0.10358589]),
 array([0.09726162]),
 array([0.09649343]),
 array([0.09978062]),
 array([0.10449346]),
 array([0.10745908]),
 array([0.11049616]),
 array([0.11125721]),
 array([0.11132511]),
 array([0.11011026]),
 array([0.11060692]),
 array([0.107284]),
 array([0.10112051]),
 array([0.09813344]),
 array([0.10390033]),
 array([0.09881948]),
 array([0.09606466]),
 array([0.09675068]),
 array([0.10037373]),
 array([0.09871227]),
 array([0.09363499]),
 array([0.08848268]),
 array([0.08765374]),
 array([0.08946169]),
 array([0.08488462]),
 array([0.08545989]),
 array([0.08726427]),
 array([0.09263454]),
 array([0.09078371]),
 array([0.08650679]),
 array([0.07836027]),
 array([0.08700344]),
 array([0.07961797]),
 array([0.08085781]),
 array([0.07825307]),
 array([0.07627361]),
 array([0.07317222]),
 array([0.07360813]),
 array([0.06655495]),
 array([0.06934906]),
 array([0.06871663]),
 array([0.05983764]),
 array([0.05260224]),
 array([0.05592874]),
 array([0.05441734]),
 array([0.05444236]),
 array([0.05229139]),
 array([0.04826815]),
 array([0.0490042]),
 array([0.0625996]),
 array([0.06087739]),
 array([0.06115967]),
 array([0.06466125]),
 array([0.07110343]),
 array([0.06940624]),
 array([0.0687488]),
 array([0.07104268]),
 array([0.06958488]),
 array([0.06991004]),
 array([0.0662155]),
 array([0.05297384]),
 array([0.05599304]),
 array([0.05099438]),
 array([0.04976882]),
 array([0.05390283]),
 array([0.05304887]),
 array([0.04972237]),
 array([0.04261203]),
 array([0.04584205]),
 array([0.05122663]),
 array([0.04851468]),
 array([0.04687824]),
 array([0.04602071]),
 array([0.04632084]),
 array([0.04375897]),
 array([0.04449503]),
 array([0.04254056]),
 array([0.05060848]),
 array([0.05663262]),
 array([0.05415651]),
 array([0.04876122]),
 array([0.04765359]),
 array([0.04815739]),
 array([0.04522392]),
 array([0.04751423]),
 array([0.0463673]),
 array([0.03973932]),
 array([0.0340832]),
 array([0.04129]),
 array([0.04007161]),
 array([0.03911404]),
 array([0.04081837]),
 array([0.04411985]),
 array([0.02142749]),
 array([0.01763293]),
 array([0.02118811]),
 array([0.02420374]),
 array([0.02368923]),
 array([0.02321044]),
 array([0.02254228]),
 array([0.01850476]),
 array([0.02405011]),
 array([0.02387503]),
 array([0.02775892]),
 array([0.03017429]),
 array([0.03194295]),
 array([0.02764458]),
 array([0.02732658]),
 array([0.02717652]),
 array([0.02487905]),
 array([0.02481832]),
 array([0.02083795]),
 array([0.01984107]),
 array([0.02153826]),
 array([0.01867626]),
 array([0.02088083]),
 array([0.01930868]),
 array([0.01817604]),
 array([0.01427071]),
 array([0.0105476]),
 array([0.0145101]),
 array([0.01255207]),
 array([0.01431001]),
 array([0.01471734]),
 array([0.01691475]),
 array([0.0135418]),
 array([0.01351322]),
 array([0.01499603]),
 array([0.01898354]),
 array([0.02329263]),
 array([0.02285313]),
 array([0.02199203]),
 array([0.02222428]),
 array([0.02550434]),
 array([0.02610103]),
 array([0.02522921]),
 array([0.02199203]),
 array([0.01862624]),
 array([0.01371331]),
 array([0.01402774]),
 array([0.0148138]),
 array([0.01328812]),
 array([0.01167311]),
 array([0.01274859]),
 array([0.01302372]),
 array([0.01613583]),
 array([0.0156499]),
 array([0.01403132]),
 array([0.01047614]),
 array([0.01275931]),
 array([0.00438412]),
 array([0.00054309]),
 array([0.]),
 array([0.00290845]),
 array([0.00557393]),
 array([0.00533454]),
 array([0.00637787]),
 array([0.00952928]),
 array([0.01414565]),
 array([0.01866911]),
 array([0.01742212]),
 array([0.01964812]),
 array([0.02124171]),
 array([0.02507556]),
 array([0.0243431]),
 array([0.02619393]),
 array([0.02366778]),
 array([0.02231004]),
 array([0.02294247]),
 array([0.01905501]),
 array([0.01369187]),
 array([0.01573923]),
 array([0.01526759]),
 array([0.01872271]),
 array([0.01755433]),
 array([0.01815817]),
 array([0.01844044]),
 array([0.01951592]),
 array([0.01819033]),
 array([0.01944446]),
 array([0.02181339]),
 array([0.02115238]),
 array([0.02150611]),
 array([0.0210023]),
 array([0.01950163]),
 array([0.01712556]),
 array([0.01832253]),
 array([0.0172792]),
 array([0.01681828]),
 array([0.01488527]),
 array([0.0162323]),
 array([0.01412064]),
 array([0.01481738]),
 array([0.0147352]),
 array([0.01160164]),
 array([0.00940066]),
 array([0.00820727]),
 array([0.00429121]),
 array([0.00432337]),
 array([0.00269406]),
 array([0.00116123]),
 array([0.00214382]),
 array([0.006678]),
 array([0.00999021]),
 array([0.01081558]),
 array([0.00960789]),
 array([0.00876109]),
 array([0.01136939]),
 array([0.01079057]),
 array([0.01313448]),
 array([0.01285578]),
 array([0.01318808]),
 array([0.01417423]),
 array([0.01421354]),
 array([0.01473163]),
 array([0.01229839]),
 array([0.01278432]),
 array([0.01016886]),
 array([0.01785803]),
 array([0.01713985]),
 array([0.01802954]),
 array([0.02045921]),
 array([0.0224351]),
 array([0.02215282]),
 array([0.02363563]),
 array([0.02572943]),
 array([0.0281984]),
 array([0.02669773]),
 array([0.02660125]),
 array([0.02518277]),
 array([0.02283884]),
 array([0.02745164]),
 array([0.03538734]),
 array([0.03857808]),
 array([0.03836726]),
 array([0.03994655]),
 array([0.04187956]),
 array([0.03949634]),
 array([0.03995727]),
 array([0.04017164]),
 array([0.03947848]),
 array([0.04017523]),
 array([0.03503719]),
 array([0.03586256]),
 array([0.03614841]),
 array([0.03454769]),
 array([0.03503362]),
 array([0.03864597]),
 array([0.03742397]),
 array([0.03847803]),
 array([0.04131859]),
 array([0.03719888]),
 array([0.02757669]),
 array([0.02935607]),
 array([0.02657267]),
 array([0.02129172]),
 array([0.0231497]),
 array([0.02649407]),
 array([0.02921672]),
 array([0.0274695]),
 array([0.03576967]),
 array([0.03521942]),
 array([0.03251463]),
 array([0.03419038]),
 array([0.03295054]),
 array([0.03080671]),
 array([0.03481209]),
 array([0.03538377]),
 array([0.03318636]),
 array([0.03305059]),
 array([0.03473707]),
 array([0.03230381]),
 array([0.03432258]),
 array([0.03541236]),
 array([0.03654501]),
 array([0.0376991]),
 array([0.03864238]),
 array([0.03951064]),
 array([0.04072189]),
 array([0.04229045]),
 array([0.04674604]),
 array([0.04621365]),
 array([0.04803233]),
 array([0.05051559]),
 array([0.04838964]),
 array([0.04979027]),
 array([0.04507386]),
 array([0.0480109]),
 array([0.04722481]),
 array([0.04627082]),
 array([0.0486719]),
 array([0.04820741]),
 array([0.04658883]),
 array([0.04357676]),
 array([0.04646019]),
 array([0.04592067]),
 array([0.04626367]),
 array([0.04648521]),
 array([0.04917571]),
 array([0.04804305]),
 array([0.04577059]),
 array([0.04609931]),
 array([0.04447358]),
 array([0.04666742]),
 array([0.04618864]),
 array([0.04759642]),
 array([0.05104796]),
 array([0.05553571]),
 array([0.05914804]),
 array([0.05741871]),
 array([0.0628104]),
 array([0.06233877]),
 array([0.06337495]),
 array([0.06055938]),
 array([0.06284971]),
 array([0.06253528]),
 array([0.06103817]),
 array([0.06074519]),
 array([0.05856207]),
 array([0.059659]),
 array([0.05876216]),
 array([0.05725433]),
 array([0.05499975]),
 array([0.05662905]),
 array([0.06415744]),
 array([0.06329276]),
 array([0.06194574]),
 array([0.0605844]),
 array([0.05859422]),
 array([0.0609167]),
 array([0.05809757]),
 array([0.05375634]),
 array([0.05481038]),
 array([0.05342048]),
 array([0.05464245]),
 array([0.05216276]),
 array([0.05088361]),
 array([0.0518805]),
 array([0.05568935]),
 array([0.05960896]),
 array([0.05849776]),
 array([0.05364557]),
 array([0.05664691]),
 array([0.05751873]),
 array([0.05918735]),
 array([0.05557501]),
 array([0.05715787]),
 array([0.0414365]),
 array([0.039382]),
 array([0.03903542]),
 array([0.0393284]),
 array([0.0396607]),
 array([0.04225473]),
 array([0.04361248]),
 array([0.0435839]),
 array([0.04614576]),
 array([0.04947226]),
 array([0.05196266]),
 array([0.05194837]),
 array([0.05498902]),
 array([0.05483182]),
 array([0.05554643]),
 array([0.05246647]),
 array([0.05024404]),
 array([0.04813594]),
 array([0.04895775]),
 array([0.04699615]),
 array([0.04531325]),
 array([0.04900063]),
 array([0.04848967]),
 array([0.04903278]),
 array([0.05027619]),
 array([0.05067638]),
 array([0.05010111]),
 array([0.04999036]),
 array([0.05016185]),
 array([0.05200911]),
 array([0.05219491]),
 array([0.05006539]),
 array([0.04793585]),
 array([0.04866833]),
 array([0.05033336]),
 array([0.05028334]),
 array([0.04936865]),
 array([0.05085859]),
 array([0.05311676]),
 array([0.05518912]),
 array([0.05332756]),
 array([0.05249862]),
 array([0.05228423]),
 array([0.05224136]),
 array([0.05399573]),
 array([0.05431729]),
 array([0.05297384]),
 array([0.05048344]),
 array([0.04749994]),
 array([0.04748923]),
 array([0.04994748]),
 array([0.04750353]),
 array([0.04612076]),
 array([0.04686036]),
 array([0.0455312]),
 array([0.04590637]),
 array([0.04802519]),
 array([0.05025119]),
 array([0.05044056]),
 array([0.0479573]),
 array([0.0633285]),
 array([0.06481845]),
 array([0.07273274]),
 array([0.07210387]),
 array([0.07130351]),
 array([0.07180018]),
 array([0.07219321]),
 array([0.0751874]),
 array([0.07284708]),
 array([0.07210387]),
 array([0.07055318]),
 array([0.06967779]),
 array([0.07228253]),
 array([0.07261483]),
 array([0.07265412]),
 array([0.07084974]),
 array([0.07395472]),
 array([0.07648443]),
 array([0.0765273]),
 array([0.07709898]),
 array([0.07744198]),
 array([0.0798931]),
 array([0.08400209]),
 array([0.08342326]),
 array([0.08748579]),
 array([0.08663542]),
 array([0.08508115]),
 array([0.08825757]),
 array([0.09085874]),
 array([0.09176272]),
 array([0.09112673]),
 array([0.09481766]),
 array([0.09619329]),
 array([0.09521785]),
 array([0.09129108]),
 array([0.08876494]),
 array([0.09106597]),
 array([0.09076585]),
 array([0.09101596]),
 array([0.0902156]),
 array([0.08783954]),
 array([0.08763944]),
 array([0.08626382]),
 array([0.08646391]),
 array([0.08781452]),
 array([0.09051574]),
 array([0.0928918]),
 array([0.09436746]),
 array([0.09426743]),
 array([0.09564304]),
 array([0.10049523]),
 array([0.09894452]),
 array([0.09904457]),
 array([0.09816918]),
 array([0.09861938]),
 array([0.10169576]),
 array([0.09886949]),
 array([0.09751888]),
 array([0.09329197]),
 array([0.09664349]),
 array([0.09541794]),
 array([0.09736882]),
 array([0.1035466]),
 array([0.10314642]),
 array([0.10474713]),
 array([0.10812365]),
 array([0.10652293]),
 array([0.10594768]),
 array([0.0995698]),
 array([0.1008954]),
 array([0.09954479]),
 array([0.09836928]),
 array([0.09796909]),
 array([0.09676855]),
 array([0.09741883]),
 array([0.10054524]),
 array([0.10049523]),
 array([0.10367165]),
 array([0.10432195]),
 array([0.10552249]),
 array([0.10847383]),
 array([0.11190035]),
 array([0.1120004]),
 array([0.11202542]),
 array([0.11387625]),
 array([0.1144265]),
 array([0.11280076]),
 array([0.11590215]),
 array([0.11620229]),
 array([0.11682757]),
 array([0.11882848]),
 array([0.10792357]),
 array([0.10587265]),
 array([0.1079986]),
 array([0.10647292]),
 array([0.10554749]),
 array([0.11307589]),
 array([0.11415137]),
 array([0.11472664]),
 array([0.11465159]),
 array([0.11272573]),
 array([0.11452654]),
 array([0.11505178]),
 array([0.11297584]),
 array([0.11322595]),
 array([0.11717773]),
 array([0.11495173]),
 array([0.10524737]),
 array([0.1124506]),
 array([0.11084989]),
 array([0.1124506]),
 array([0.10852384]),
 array([0.11032465]),
 array([0.10962434]),
 array([0.10962434]),
 array([0.10744835]),
 array([0.11257567]),
 array([0.1131259]),
 array([0.11240059]),
 array([0.11009954]),
 array([0.10744835]),
 array([0.10442199]),
 array([0.10122056]),
 array([0.10474713]),
 array([0.1099745]),
 array([0.11675254]),
 array([0.11805312]),
 array([0.1226552]),
 array([0.12363064]),
 array([0.12335551]),
 array([0.12743234]),
 array([0.12893301]),
 array([0.12803262]),
 array([0.13058377]),
 array([0.13408535]),
 array([0.13208444]),
 array([0.13273474]),
 array([0.13233455]),
 array([0.13310991]),
 array([0.1326597]),
 array([0.13483567]),
 array([0.13871242]),
 array([0.14263919]),
 array([0.14604072]),
 array([0.1455655]),
 array([0.14926718]),
 array([0.14726627]),
 array([0.15136811]),
 array([0.1517683]),
 array([0.15717072]),
 array([0.15459457]),
 array([0.15809615]),
 array([0.15792107]),
 array([0.14826672]),
 array([0.14716622]),
 array([0.15041769]),
 array([0.14931719]),
 array([0.14809164]),
 array([0.14158872]),
 array([0.14589066]),
 array([0.1404632]),
 array([0.13963785]),
 array([0.13491072]),
 array([0.13115903]),
 array([0.12745736]),
 array([0.13411036]),
 array([0.142214]),
 array([0.14003802]),
 array([0.14293933]),
 array([0.14193888]),
 array([0.14061328]),
 array([0.1455655]),
 array([0.14536543]),
 array([0.14188885]),
 array([0.13653644]),
 array([0.13391027]),
 array([0.1262068]),
 array([0.12623181]),
 array([0.12995848]),
 array([0.14031314]),
 array([0.14061328]),
 array([0.13371017]),
 array([0.13613627]),
 array([0.1350858]),
 array([0.12763244]),
 array([0.1255565]),
 array([0.13238458]),
 array([0.13446052]),
 array([0.14158872]),
 array([0.14303938]),
 array([0.1433395]),
 array([0.13343505]),
 array([0.14886699]),
 array([0.15784604]),
 array([0.15349409]),
 array([0.15717072]),
 array([0.15722076]),
 array([0.15949677]),
 array([0.16044721]),
 array([0.15792107]),
 array([0.15989696]),
 array([0.16564954]),
 array([0.17280276]),
 array([0.17675454]),
 array([0.17830525]),
 array([0.18018109]),
 array([0.18240709]),
 array([0.18173178]),
 array([0.18435797]),
 array([0.19311191]),
 array([0.19103597]),
 array([0.18258215]),
 array([0.186659]),
 array([0.18175681]),
 array([0.18333249]),
 array([0.18400781]),
 array([0.18195687]),
 array([0.1766295]),
 array([0.1771047]),
 array([0.17845531]),
 array([0.17187735]),
 array([0.16619979]),
 array([0.17172727]),
 array([0.1695763]),
 array([0.17297784]),
 array([0.1782052]),
 array([0.18178181]),
 array([0.17935571]),
 array([0.17535392]),
 array([0.17863039]),
 array([0.17732981]),
 array([0.16905107]),
 array([0.17120204]),
 array([0.16872593]),
 array([0.17652945]),
 array([0.17167726]),
 array([0.17122706]),
 array([0.17390326]),
 array([0.17898054]),
 array([0.17562904]),
 array([0.17460357]),
 array([0.17700465]),
 array([0.17835526]),
 array([0.17772998]),
 array([0.17635437]),
 array([0.1775549]),
 array([0.17602921]),
 array([0.17247762]),
 array([0.17960582]),
 array([0.17788006]),
 array([0.18215696]),
 array([0.18478315]),
 array([0.18630884]),
 array([0.1922365]),
 array([0.18700915]),
 array([0.182207]),
 array([0.17347807]),
 array([0.18298234]),
 array([0.18235706]),
 array([0.17510381]),
 array([0.17312792]),
 array([0.1737532]),
 array([0.17965586]),
 array([0.17640439]),
 array([0.17527889]),
 array([0.17562904]),
 array([0.18298234]),
 array([0.18253215]),
 array([0.18608375]),
 array([0.18578363]),
 array([0.18575859]),
 array([0.18908509]),
 array([0.19196137]),
 array([0.18465809]),
 array([0.19071081]),
 array([0.19006053]),
 array([0.18630884]),
 array([0.18695912]),
 array([0.1855085]),
 array([0.18590865]),
 array([0.18400781]),
 array([0.18223199]),
 array([0.18010606]),
 array([0.17915562]),
 array([0.18280728]),
 array([0.18208193]),
 array([0.17853034]),
 array([0.17790506]),
 array([0.17960582]),
 array([0.1788555]),
 array([0.18030613]),
 array([0.1771047]),
 array([0.17963084]),
 array([0.17818018]),
 array([0.1808814]),
 array([0.17935571]),
 array([0.17747987]),
 array([0.17192737]),
 array([0.17417838]),
 array([0.1771047]),
 array([0.17670453]),
 array([0.17560403]),
 array([0.17482868]),
 array([0.16702517]),
 array([0.16077235]),
 array([0.16880096]),
 array([0.17475365]),
 array([0.17462859]),
 array([0.17765495]),
 array([0.18188184]),
 array([0.18465809]),
 array([0.19078588]),
 array([0.18748437]),
 array([0.17365315]),
 array([0.17350309]),
 array([0.17185234]),
 array([0.16752538]),
 array([0.16905107]),
 array([0.16807563]),
 array([0.16652495]),
 array([0.16384874]),
 array([0.15669552]),
 array([0.14719124]),
 array([0.1490921]),
 array([0.14841678]),
 array([0.14939222]),
 array([0.15989696]),
 array([0.14431494]),
 array([0.14869191]),
 array([0.14846682]),
 array([0.15049272]),
 array([0.15349409]),
 array([0.15184333]),
 array([0.14811666]),
 array([0.142214]),
 array([0.12498125]),
 array([0.11837828]),
 array([0.11992897]),
 array([0.13481067]),
 array([0.1428893]),
 array([0.14381472]),
 array([0.14248913]),
 array([0.12988345]),
 array([0.14143864]),
 array([0.13651144]),
 array([0.13376019]),
 array([0.14136361]),
 array([0.13596119]),
 array([0.14201391]),
 array([0.14611575]),
 array([0.14886699]),
 array([0.15129308]),
 array([0.15161824]),
 array([0.14539042]),
 array([0.14421489]),
 array([0.14861688]),
 array([0.14408985]),
 array([0.14639088]),
 array([0.14809164]),
 array([0.14736632]),
 array([0.14168877]),
 array([0.13323495]),
 array([0.13633636]),
 array([0.13453555]),
 array([0.13653644]),
 array([0.13753689]),
 array([0.13886249]),
 array([0.13753689]),
 array([0.13433545]),
 array([0.14088841]),
 array([0.13958781]),
 array([0.14006303]),
 array([0.13611125]),
 array([0.14023811]),
 array([0.13818719]),
 array([0.13991297]),
 array([0.14501525]),
 array([0.14499025]),
 array([0.14934221]),
 array([0.15829624]),
 array([0.14879196]),
 array([0.14696615]),
 array([0.15877144]),
 array([0.16192287]),
 array([0.15934671]),
 array([0.1635486]),
 array([0.16702517]),
 array([0.16559952]),
 array([0.1628983]),
 array([0.16324846]),
 array([0.16202291]),
 array([0.15251863]),
 array([0.1508679]),
 array([0.14989246]),
 array([0.14143864]),
 array([0.14604072]),
 array([0.14481518]),
 array([0.15381922]),
 array([0.1575459]),
 array([0.15884649]),
 array([0.15496974]),
 array([0.157796]),
 array([0.15567005]),
 array([0.1551198]),
 array([0.15634537]),
 array([0.15394427]),
 array([0.15129308]),
 array([0.14859186]),
 array([0.15817118]),
 array([0.15629533]),
 array([0.15617029]),
 array([0.14964235]),
 array([0.15101796]),
 array([0.1435396]),
 array([0.14178882]),
 array([0.13681156]),
 array([0.13893751]),
 array([0.13303488]),
 array([0.12565655]),
 array([0.12890802]),
 array([0.12865791]),
 array([0.13210945]),
 array([0.1306588]),
 array([0.12763244]),
 array([0.13243459]),
 array([0.128883]),
 array([0.12373069]),
 array([0.12395578]),
 ...]
In [49]:
# Converted the data into array format
X = np.asarray(X)
y = np.asarray(y)
In [50]:
# Splitted the data
split = int(0.7 * len(X))
X_train = X[:split]
y_train = y[:split]
X_test = X[split:]
y_test = y[split:]
In [51]:
# Reshaped the 1D arrays to 3D arrays to feed in the model
X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))
X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))
X_train.shape, X_test.shape
Out[51]:
((1510, 1, 1), (648, 1, 1))
In [52]:
# Created the model
inputs = keras.layers.Input(shape=(X_train.shape[1], X_train.shape[2]))
x = keras.layers.LSTM(150, return_sequences= True)(inputs)
x = keras.layers.Dropout(0.3)(x)
x = keras.layers.LSTM(150, return_sequences=True)(x)
x = keras.layers.Dropout(0.3)(x)
x = keras.layers.LSTM(150)(x)
outputs = keras.layers.Dense(1, activation='linear')(x)

model = keras.Model(inputs=inputs, outputs=outputs)
model.compile(optimizer='adam', loss="mse")
model.summary()
Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         [(None, 1, 1)]            0         
_________________________________________________________________
lstm (LSTM)                  (None, 1, 150)            91200     
_________________________________________________________________
dropout (Dropout)            (None, 1, 150)            0         
_________________________________________________________________
lstm_1 (LSTM)                (None, 1, 150)            180600    
_________________________________________________________________
dropout_1 (Dropout)          (None, 1, 150)            0         
_________________________________________________________________
lstm_2 (LSTM)                (None, 150)               180600    
_________________________________________________________________
dense (Dense)                (None, 1)                 151       
=================================================================
Total params: 452,551
Trainable params: 452,551
Non-trainable params: 0
_________________________________________________________________
In [53]:
# Trained the model
history = model.fit(
    X_train, y_train,
    epochs = 20,
    batch_size = 32,
    validation_split = 0.2
)
Epoch 1/20
38/38 [==============================] - 13s 111ms/step - loss: 0.0064 - val_loss: 0.0212
Epoch 2/20
38/38 [==============================] - 0s 12ms/step - loss: 0.0021 - val_loss: 0.0057
Epoch 3/20
38/38 [==============================] - 0s 12ms/step - loss: 5.2621e-04 - val_loss: 3.5259e-05
Epoch 4/20
38/38 [==============================] - 0s 12ms/step - loss: 4.9535e-05 - val_loss: 2.4259e-05
Epoch 5/20
38/38 [==============================] - 0s 12ms/step - loss: 5.1024e-05 - val_loss: 2.0410e-05
Epoch 6/20
38/38 [==============================] - 0s 12ms/step - loss: 4.2864e-05 - val_loss: 1.7693e-05
Epoch 7/20
38/38 [==============================] - 0s 12ms/step - loss: 3.8262e-05 - val_loss: 3.5504e-05
Epoch 8/20
38/38 [==============================] - 0s 13ms/step - loss: 3.8512e-05 - val_loss: 2.3805e-05
Epoch 9/20
38/38 [==============================] - 0s 12ms/step - loss: 3.5577e-05 - val_loss: 2.2485e-05
Epoch 10/20
38/38 [==============================] - 1s 18ms/step - loss: 4.8163e-05 - val_loss: 1.9575e-05
Epoch 11/20
38/38 [==============================] - 0s 12ms/step - loss: 4.0308e-05 - val_loss: 2.1232e-05
Epoch 12/20
38/38 [==============================] - 0s 12ms/step - loss: 3.8491e-05 - val_loss: 3.9104e-05
Epoch 13/20
38/38 [==============================] - 0s 12ms/step - loss: 3.7558e-05 - val_loss: 3.7024e-05
Epoch 14/20
38/38 [==============================] - 0s 12ms/step - loss: 3.4061e-05 - val_loss: 2.1085e-05
Epoch 15/20
38/38 [==============================] - 0s 12ms/step - loss: 3.3370e-05 - val_loss: 1.6435e-05
Epoch 16/20
38/38 [==============================] - 0s 12ms/step - loss: 3.9214e-05 - val_loss: 7.5378e-05
Epoch 17/20
38/38 [==============================] - 0s 12ms/step - loss: 4.1332e-05 - val_loss: 1.7440e-05
Epoch 18/20
38/38 [==============================] - 0s 12ms/step - loss: 3.6411e-05 - val_loss: 1.9102e-05
Epoch 19/20
38/38 [==============================] - 0s 12ms/step - loss: 3.4991e-05 - val_loss: 1.7209e-05
Epoch 20/20
38/38 [==============================] - 0s 13ms/step - loss: 4.2519e-05 - val_loss: 3.2051e-05
In [54]:
# Made prediction
predicted = model.predict(X)
In [55]:
# Append the predicted values to the list
test_predicted = []

for i in predicted:
  test_predicted.append(i[0])
In [56]:
test_predicted
Out[56]:
[0.009663401,
 0.009108094,
 0.010826838,
 0.012377107,
 0.011898997,
 0.009280309,
 0.011779472,
 0.009318966,
 0.01854866,
 0.017834624,
 0.018766755,
 0.020782504,
 0.022003379,
 0.02190134,
 0.021524867,
 0.023129381,
 0.024639089,
 0.02634964,
 0.029113064,
 0.03492008,
 0.035008144,
 0.03824214,
 0.040659353,
 0.036505245,
 0.03810472,
 0.03807302,
 0.04255894,
 0.04192101,
 0.043101713,
 0.045223735,
 0.046404764,
 0.049807355,
 0.052286714,
 0.053002708,
 0.053253166,
 0.049013935,
 0.04799135,
 0.048142977,
 0.052127972,
 0.053249642,
 0.05565904,
 0.061340272,
 0.06892347,
 0.067503974,
 0.0675075,
 0.07299204,
 0.0747088,
 0.07348656,
 0.07237039,
 0.07120839,
 0.07506912,
 0.07771889,
 0.07882838,
 0.07608658,
 0.07244457,
 0.07918528,
 0.08296316,
 0.081192486,
 0.08450425,
 0.08540566,
 0.08265214,
 0.08186045,
 0.08064825,
 0.07445092,
 0.06558681,
 0.07603005,
 0.075549595,
 0.06816779,
 0.06306273,
 0.06261092,
 0.058580533,
 0.07613605,
 0.07532348,
 0.07366319,
 0.06694609,
 0.06629292,
 0.06765227,
 0.06618346,
 0.06033442,
 0.06182734,
 0.06136851,
 0.061721466,
 0.062194407,
 0.060849696,
 0.05785363,
 0.056071825,
 0.05357063,
 0.047941998,
 0.048033662,
 0.0589334,
 0.057412572,
 0.062208533,
 0.060359128,
 0.05928982,
 0.062812105,
 0.06524788,
 0.064739525,
 0.058831073,
 0.05999562,
 0.0594804,
 0.062526226,
 0.06261798,
 0.06565388,
 0.062423848,
 0.06418527,
 0.062773295,
 0.06255092,
 0.06346868,
 0.067581646,
 0.06815722,
 0.06756752,
 0.064718336,
 0.06628233,
 0.062282674,
 0.062727414,
 0.06359928,
 0.061675556,
 0.06695317,
 0.069961704,
 0.0723951,
 0.07611484,
 0.07468053,
 0.07751042,
 0.07550366,
 0.07416833,
 0.07221498,
 0.074359074,
 0.07504441,
 0.075054996,
 0.07481479,
 0.077662356,
 0.07412241,
 0.07395637,
 0.07292845,
 0.063765205,
 0.06373344,
 0.067362726,
 0.070848145,
 0.07640455,
 0.07500906,
 0.07535528,
 0.07814995,
 0.08057052,
 0.07999096,
 0.079619914,
 0.07992735,
 0.08027013,
 0.08320352,
 0.08380086,
 0.08349687,
 0.085444555,
 0.08960575,
 0.09563195,
 0.092416994,
 0.09694778,
 0.09474062,
 0.0949493,
 0.0993569,
 0.099045575,
 0.09857505,
 0.095179185,
 0.09566378,
 0.099105716,
 0.097428896,
 0.09956563,
 0.10104098,
 0.094779514,
 0.0940191,
 0.09727323,
 0.1019397,
 0.10487676,
 0.10788505,
 0.108638965,
 0.10870621,
 0.10750277,
 0.10799477,
 0.10470335,
 0.09859982,
 0.09564254,
 0.10135234,
 0.0963217,
 0.093594685,
 0.09427373,
 0.09786045,
 0.09621556,
 0.0911899,
 0.086091466,
 0.08527133,
 0.087060116,
 0.08253196,
 0.083101004,
 0.08488602,
 0.09019978,
 0.088368244,
 0.08413664,
 0.076079525,
 0.08462797,
 0.07732315,
 0.07854922,
 0.075973526,
 0.07401642,
 0.07095057,
 0.07138144,
 0.06441121,
 0.06717207,
 0.06654712,
 0.057776004,
 0.050632574,
 0.053916328,
 0.052424252,
 0.052448943,
 0.050325766,
 0.04635539,
 0.04708168,
 0.060503826,
 0.058802836,
 0.05908163,
 0.06254034,
 0.06890582,
 0.06722856,
 0.06657891,
 0.06884578,
 0.067405105,
 0.06772643,
 0.06407583,
 0.050999366,
 0.053979814,
 0.049045675,
 0.0478362,
 0.05191636,
 0.051073425,
 0.047790363,
 0.040775646,
 0.043961756,
 0.049274884,
 0.04659865,
 0.044984035,
 0.04413802,
 0.044434108,
 0.04190691,
 0.042632952,
 0.040705167,
 0.04866483,
 0.054611266,
 0.052166775,
 0.04684193,
 0.045749016,
 0.046246115,
 0.043351978,
 0.04561152,
 0.044479944,
 0.037942663,
 0.032366626,
 0.03947182,
 0.038270324,
 0.037326105,
 0.03900672,
 0.04226288,
 0.01989946,
 0.016164035,
 0.01966377,
 0.022633217,
 0.022126535,
 0.021655055,
 0.020997113,
 0.017022165,
 0.022481926,
 0.022309506,
 0.026134923,
 0.028514557,
 0.03025735,
 0.026022289,
 0.025709033,
 0.025561212,
 0.023298293,
 0.023238473,
 0.019319024,
 0.018337613,
 0.020008525,
 0.017190976,
 0.019361243,
 0.01781352,
 0.0166986,
 0.012855239,
 0.009192437,
 0.013090791,
 0.011164293,
 0.012893909,
 0.013294713,
 0.015457179,
 0.012138046,
 0.01210992,
 0.01356895,
 0.017493457,
 0.02173598,
 0.021303203,
 0.020455308,
 0.020683989,
 0.023914129,
 0.024501842,
 0.023643157,
 0.020455308,
 0.017141744,
 0.012306796,
 0.012616169,
 0.013389636,
 0.011888448,
 0.010299585,
 0.01135763,
 0.011628311,
 0.014690598,
 0.014212392,
 0.012619683,
 0.009122156,
 0.011368182,
 0.003131505,
 -0.0006438624,
 -0.001177568,
 0.0016809013,
 0.0043012593,
 0.00406589,
 0.005091712,
 0.008190833,
 0.012732184,
 0.017183945,
 0.015956545,
 0.01814767,
 0.019716542,
 0.02349183,
 0.02277046,
 0.024593342,
 0.02210541,
 0.020768426,
 0.021391176,
 0.017563801,
 0.0122857,
 0.014300302,
 0.0138361715,
 0.017236706,
 0.016086666,
 0.016681012,
 0.016958853,
 0.018017527,
 0.016712671,
 0.017947178,
 0.020279411,
 0.0196286,
 0.019976867,
 0.019480836,
 0.01800346,
 0.015664661,
 0.016842796,
 0.015815876,
 0.015362239,
 0.013459954,
 0.01478554,
 0.012707577,
 0.013393158,
 0.013312285,
 0.010229286,
 0.00806433,
 0.006890647,
 0.003040176,
 0.0030717868,
 0.0014701728,
 -3.6386773e-05,
 0.00092933606,
 0.0053868303,
 0.008644183,
 0.009456033,
 0.008268147,
 0.0074352995,
 0.0100008175,
 0.009431437,
 0.011737284,
 0.011463089,
 0.011790022,
 0.012760309,
 0.012798986,
 0.013308773,
 0.010914715,
 0.011392782,
 0.008819899,
 0.016385594,
 0.01567873,
 0.016554412,
 0.018946148,
 0.020891571,
 0.020613626,
 0.02207375,
 0.02413583,
 0.026567865,
 0.025089588,
 0.024994554,
 0.023597416,
 0.021289127,
 0.025832228,
 0.03365209,
 0.036797658,
 0.036589794,
 0.038147002,
 0.04005325,
 0.03770308,
 0.03815758,
 0.038368963,
 0.03768546,
 0.038372498,
 0.033306934,
 0.034120526,
 0.03440231,
 0.03282445,
 0.033303414,
 0.036864597,
 0.035659794,
 0.036699012,
 0.039500013,
 0.035437886,
 0.025955409,
 0.02770839,
 0.024966396,
 0.019765783,
 0.021595236,
 0.024888974,
 0.027571093,
 0.025849821,
 0.03402896,
 0.033486556,
 0.030820722,
 0.03247227,
 0.031250313,
 0.029137697,
 0.033085056,
 0.033648565,
 0.03148272,
 0.031348918,
 0.03301111,
 0.030612964,
 0.03260257,
 0.033676744,
 0.03479328,
 0.035931043,
 0.036861062,
 0.03771717,
 0.038911574,
 0.040458493,
 0.0448536,
 0.04432836,
 0.04612271,
 0.04857316,
 0.04647527,
 0.047857374,
 0.043203946,
 0.04610157,
 0.045325965,
 0.044384766,
 0.046753787,
 0.04629546,
 0.0446985,
 0.04172718,
 0.044571586,
 0.044039324,
 0.044377703,
 0.04459627,
 0.04725092,
 0.04613329,
 0.04389126,
 0.04421556,
 0.042611793,
 0.044776037,
 0.04430368,
 0.045692608,
 0.04909855,
 0.05352831,
 0.057095006,
 0.055387408,
 0.06071204,
 0.0602462,
 0.061269686,
 0.05848878,
 0.060750872,
 0.060440294,
 0.05896163,
 0.05867227,
 0.05651637,
 0.05759958,
 0.05671396,
 0.05522511,
 0.052999184,
 0.05460774,
 0.06204264,
 0.061188497,
 0.059857994,
 0.058513485,
 0.05654812,
 0.058841653,
 0.056057706,
 0.05177176,
 0.05281225,
 0.051440224,
 0.052646473,
 0.05019881,
 0.048936345,
 0.049920216,
 0.053679995,
 0.057550177,
 0.056452878,
 0.051662408,
 0.05462537,
 0.055486165,
 0.057133824,
 0.053567104,
 0.055129863,
 0.03961629,
 0.037590332,
 0.03724859,
 0.037537474,
 0.03786514,
 0.040423263,
 0.041762415,
 0.04173422,
 0.044261385,
 0.047543563,
 0.05000131,
 0.04998721,
 0.052988604,
 0.052833423,
 0.053538896,
 0.050498568,
 0.04830517,
 0.04622495,
 0.04703585,
 0.045100365,
 0.04344009,
 0.047078155,
 0.04657397,
 0.047109887,
 0.0483369,
 0.048731834,
 0.04816412,
 0.04805482,
 0.04822406,
 0.05004716,
 0.05023054,
 0.048128873,
 0.04602751,
 0.04675027,
 0.048393317,
 0.048343964,
 0.04744131,
 0.04891166,
 0.051140442,
 0.05318614,
 0.051348515,
 0.0505303,
 0.050318703,
 0.050276384,
 0.052008055,
 0.052325495,
 0.050999366,
 0.04854142,
 0.045597415,
 0.045586858,
 0.048012517,
 0.04560096,
 0.04423672,
 0.044966396,
 0.043655097,
 0.04402522,
 0.046115674,
 0.048312232,
 0.048499107,
 0.04604868,
 0.061223805,
 0.06269563,
 0.07051617,
 0.0698946,
 0.06910356,
 0.069594435,
 0.0699829,
 0.072942585,
 0.07062918,
 0.0698946,
 0.06836203,
 0.06749692,
 0.07007118,
 0.07039962,
 0.07043846,
 0.068655096,
 0.071724035,
 0.074224845,
 0.07426724,
 0.07483244,
 0.075171575,
 0.07759523,
 0.081659004,
 0.08108649,
 0.08510519,
 0.0842639,
 0.082726344,
 0.08586874,
 0.0884425,
 0.08933703,
 0.08870768,
 0.09236041,
 0.093722016,
 0.0927565,
 0.0888703,
 0.08637074,
 0.08864756,
 0.08835057,
 0.08859805,
 0.0878061,
 0.08545514,
 0.085257195,
 0.083896294,
 0.08409425,
 0.085430406,
 0.08810309,
 0.090454385,
 0.09191483,
 0.09181582,
 0.093177356,
 0.09798073,
 0.096445486,
 0.09654454,
 0.09567793,
 0.096123606,
 0.09916938,
 0.09637122,
 0.09503418,
 0.09085042,
 0.09416763,
 0.09295456,
 0.09488563,
 0.101002075,
 0.100605786,
 0.102190904,
 0.105534986,
 0.10394958,
 0.10337986,
 0.097064525,
 0.09837694,
 0.09703976,
 0.095876016,
 0.095479846,
 0.09429143,
 0.09493514,
 0.09803023,
 0.09798073,
 0.10112589,
 0.10176987,
 0.102958776,
 0.10588181,
 0.10927609,
 0.10937519,
 0.10939998,
 0.111233644,
 0.11177881,
 0.11016812,
 0.11324093,
 0.113538355,
 0.11415793,
 0.11614075,
 0.10533682,
 0.103305556,
 0.10541113,
 0.10390004,
 0.10298353,
 0.11044068,
 0.11150623,
 0.1120762,
 0.11200184,
 0.11009379,
 0.11187795,
 0.11239834,
 0.11034156,
 0.11058936,
 0.1145049,
 0.11229923,
 0.102686316,
 0.10982122,
 0.108235456,
 0.10982122,
 0.10593137,
 0.10771515,
 0.107021436,
 0.107021436,
 0.10486614,
 0.10994511,
 0.11049024,
 0.10977167,
 0.10749216,
 0.10486614,
 0.10186893,
 0.098698884,
 0.102190904,
 0.1073683,
 0.11408358,
 0.115372375,
 0.11993338,
 0.12090024,
 0.120627545,
 0.124668926,
 0.12615675,
 0.12526408,
 0.12779346,
 0.13126566,
 0.12928148,
 0.12992632,
 0.12952949,
 0.13029838,
 0.12985192,
 0.13200976,
 0.13585469,
 0.13974983,
 0.14312445,
 0.14265296,
 0.14632574,
 0.14434038,
 0.14841051,
 0.14880763,
 0.15416929,
 0.15161243,
 0.15508778,
 0.154914,
 0.14533304,
 0.14424114,
 0.14746739,
 0.14637539,
 0.14515936,
 0.13870776,
 0.14297558,
 0.1375913,
 0.1367726,
 0.13208419,
 0.12836386,
 0.124693744,
 0.13129048,
 0.13932806,
 0.13716954,
 0.14004758,
 0.13905512,
 0.13774018,
 0.14265296,
 0.14245448,
 0.13900548,
 0.1336965,
 0.13109204,
 0.12345397,
 0.123478755,
 0.12717348,
 0.13744244,
 0.13774018,
 0.13089362,
 0.1332996,
 0.13225782,
 0.124867305,
 0.1228093,
 0.1295791,
 0.13163772,
 0.13870776,
 0.14014684,
 0.14044456,
 0.13062078,
 0.14592865,
 0.15483953,
 0.15052028,
 0.15416929,
 0.15421891,
 0.15647802,
 0.15742147,
 0.154914,
 0.15687527,
 0.16258588,
 0.16968827,
 0.17361256,
 0.17515261,
 0.17701559,
 0.17922647,
 0.17855573,
 0.18116418,
 0.18986002,
 0.1877977,
 0.17940035,
 0.18344979,
 0.1785806,
 0.18014562,
 0.18081637,
 0.17877929,
 0.17348841,
 0.1739603,
 0.17530163,
 0.16876937,
 0.1631322,
 0.16862036,
 0.16648458,
 0.16986215,
 0.17505322,
 0.17860542,
 0.17619587,
 0.17222165,
 0.17547552,
 0.17418388,
 0.16596307,
 0.1680988,
 0.16564026,
 0.17338905,
 0.1685707,
 0.16812363,
 0.17078108,
 0.17582327,
 0.17249486,
 0.17147654,
 0.17386097,
 0.17520228,
 0.17458127,
 0.17321517,
 0.17440742,
 0.17289226,
 0.16936542,
 0.17644426,
 0.17473036,
 0.17897806,
 0.18158647,
 0.18310195,
 0.18899034,
 0.1837976,
 0.1790277,
 0.17035888,
 0.17979781,
 0.17917678,
 0.17197329,
 0.17001116,
 0.1706321,
 0.17649396,
 0.17326483,
 0.17214715,
 0.17249486,
 0.17979781,
 0.17935072,
 0.18287838,
 0.18258026,
 0.18255539,
 0.18585974,
 0.18871701,
 0.18146229,
 0.18747468,
 0.1868287,
 0.18310195,
 0.18374789,
 0.18230699,
 0.18270445,
 0.18081637,
 0.17905258,
 0.1769411,
 0.17599712,
 0.17962395,
 0.17890352,
 0.17537615,
 0.17475517,
 0.17644426,
 0.17569906,
 0.17713979,
 0.1739603,
 0.1764691,
 0.1750284,
 0.17771114,
 0.17619587,
 0.17433289,
 0.16881904,
 0.17105429,
 0.1739603,
 0.17356291,
 0.17247003,
 0.17170009,
 0.1639516,
 0.1577442,
 0.16571476,
 0.17162557,
 0.17150138,
 0.17450678,
 0.17870478,
 0.18146229,
 0.18754923,
 0.18426964,
 0.17053273,
 0.1703837,
 0.16874453,
 0.16444826,
 0.16596307,
 0.1649946,
 0.163455,
 0.16079813,
 0.15369762,
 0.14426593,
 0.14615203,
 0.14548196,
 0.14644985,
 0.15687527,
 0.14141227,
 0.14575495,
 0.14553162,
 0.14754187,
 0.15052028,
 0.14888212,
 0.14518419,
 0.13932806,
 0.12223905,
 0.1156946,
 0.117231354,
 0.13198496,
 0.13999794,
 0.140916,
 0.13960098,
 0.12709908,
 0.1385589,
 0.1336717,
 0.13094321,
 0.13848446,
 0.13312599,
 0.13912955,
 0.14319888,
 0.14592865,
 0.14833608,
 0.14865874,
 0.14247924,
 0.14131302,
 0.14568049,
 0.14118896,
 0.14347185,
 0.14515936,
 0.14443965,
 0.13880701,
 0.13042237,
 0.13349806,
 0.13171214,
 0.1336965,
 0.13468875,
 0.13600354,
 0.13468875,
 0.1315137,
 0.13801308,
 0.13672298,
 0.13719437,
 0.1332748,
 0.13736802,
 0.13533373,
 0.13704552,
 0.14210705,
 0.14208224,
 0.14640021,
 0.15528639,
 0.14585423,
 0.14404261,
 0.15575805,
 0.15888628,
 0.15632907,
 0.16050015,
 0.1639516,
 0.16253625,
 0.15985459,
 0.1602022,
 0.15898558,
 0.14955226,
 0.14791417,
 0.1469462,
 0.1385589,
 0.14312445,
 0.14190856,
 0.15084295,
 0.15454163,
 0.15583256,
 0.15198477,
 0.1547899,
 0.15267982,
 0.15213372,
 0.15335006,
 0.15096706,
 0.14833608,
 0.14565566,
 0.15516226,
 0.15330039,
 0.15317632,
 0.14669803,
 0.14806305,
 0.14064308,
 0.13890626,
 0.13396937,
 0.13607796,
 0.13022396,
 0.12290847,
 0.12613197,
 0.12588398,
 0.12930629,
 0.12786786,
 0.124867305,
 0.1296287,
 0.12610716,
 0.12099942,
 0.12122255,
 ...]
In [57]:
df_predicted = price_volume_df[1:][['Date']]
df_predicted
Out[57]:
Date
1 2012-01-13
2 2012-01-17
3 2012-01-18
4 2012-01-19
5 2012-01-20
... ...
2154 2020-08-05
2155 2020-08-06
2156 2020-08-07
2157 2020-08-10
2158 2020-08-11

2158 rows × 1 columns

In [58]:
df_predicted['predictions'] = test_predicted
In [59]:
df_predicted
Out[59]:
Date predictions
1 2012-01-13 0.009663
2 2012-01-17 0.009108
3 2012-01-18 0.010827
4 2012-01-19 0.012377
5 2012-01-20 0.011899
... ... ...
2154 2020-08-05 0.926448
2155 2020-08-06 0.930025
2156 2020-08-07 0.964375
2157 2020-08-10 0.939454
2158 2020-08-11 0.953903

2158 rows × 2 columns

In [60]:
# Plot the data
close = []
for i in training_set_scaled:
  close.append(i[0])
In [61]:
df_predicted['Close'] = close[1:]
In [62]:
df_predicted
Out[62]:
Date predictions Close
1 2012-01-13 0.009663 0.010462
2 2012-01-17 0.009108 0.012209
3 2012-01-18 0.010827 0.013785
4 2012-01-19 0.012377 0.013299
5 2012-01-20 0.011899 0.010637
... ... ... ...
2154 2020-08-05 0.926448 0.961583
2155 2020-08-06 0.930025 1.000000
2156 2020-08-07 0.964375 0.972088
2157 2020-08-10 0.939454 0.988245
2158 2020-08-11 0.953903 0.954705

2158 rows × 3 columns

In [63]:
# Plot the data
interactive_plot(df_predicted, "Original Vs Prediction")
In [64]:
# Let's test the functions and get individual stock prices and volumes for sp500
price_volume_df = individual_stock(stock_price_df, stock_vol_df, 'sp500')
price_volume_df

# Get the close and volume data as training data (Input)
training_data = price_volume_df.iloc[:, 1:3].values
training_data

# Normalize the data
sc = MinMaxScaler(feature_range = (0, 1))
training_set_scaled = sc.fit_transform(training_data)

# Created the training and testing data, training data contains present day and previous day values
X = []
y = []
for i in range(1, len(price_volume_df)):
    X.append(training_set_scaled [i-1:i, 0])
    y.append(training_set_scaled [i, 0])
    
# Converted the data into array format
X = np.asarray(X)
y = np.asarray(y)

# Splitted the data
split = int(0.7 * len(X))
X_train = X[:split]
y_train = y[:split]
X_test = X[split:]
y_test = y[split:]

# Reshaped the 1D arrays to 3D arrays to feed in the model
X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))
X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))
X_train.shape, X_test.shape

# Created the model
inputs = keras.layers.Input(shape=(X_train.shape[1], X_train.shape[2]))
x = keras.layers.LSTM(150, return_sequences= True)(inputs)
x = keras.layers.Dropout(0.3)(x)
x = keras.layers.LSTM(150, return_sequences=True)(x)
x = keras.layers.Dropout(0.3)(x)
x = keras.layers.LSTM(150)(x)
outputs = keras.layers.Dense(1, activation='linear')(x)

model = keras.Model(inputs=inputs, outputs=outputs)
model.compile(optimizer='adam', loss="mse")
model.summary()

# Trained the model
history = model.fit(
    X_train, y_train,
    epochs = 20,
    batch_size = 32,
    validation_split = 0.2
)

# Made prediction
predicted = model.predict(X)

# Append the predicted values to the list
test_predicted = []

for i in predicted:
  test_predicted.append(i[0])

test_predicted

df_predicted = price_volume_df[1:][['Date']]
df_predicted

df_predicted['predictions'] = test_predicted

df_predicted

# Plot the data
close = []
for i in training_set_scaled:
  close.append(i[0])

df_predicted['Close'] = close[1:]

df_predicted

# Plot the data
interactive_plot(df_predicted, "Original Vs Prediction")
Model: "model_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_2 (InputLayer)         [(None, 1, 1)]            0         
_________________________________________________________________
lstm_3 (LSTM)                (None, 1, 150)            91200     
_________________________________________________________________
dropout_2 (Dropout)          (None, 1, 150)            0         
_________________________________________________________________
lstm_4 (LSTM)                (None, 1, 150)            180600    
_________________________________________________________________
dropout_3 (Dropout)          (None, 1, 150)            0         
_________________________________________________________________
lstm_5 (LSTM)                (None, 150)               180600    
_________________________________________________________________
dense_1 (Dense)              (None, 1)                 151       
=================================================================
Total params: 452,551
Trainable params: 452,551
Non-trainable params: 0
_________________________________________________________________
Epoch 1/20
38/38 [==============================] - 12s 75ms/step - loss: 0.0513 - val_loss: 0.0626
Epoch 2/20
38/38 [==============================] - 0s 13ms/step - loss: 0.0088 - val_loss: 0.0048
Epoch 3/20
38/38 [==============================] - 1s 19ms/step - loss: 9.9321e-04 - val_loss: 3.4009e-04
Epoch 4/20
38/38 [==============================] - 1s 17ms/step - loss: 4.0069e-04 - val_loss: 0.0011
Epoch 5/20
38/38 [==============================] - 1s 22ms/step - loss: 3.5812e-04 - val_loss: 1.6852e-04
Epoch 6/20
38/38 [==============================] - 1s 21ms/step - loss: 3.6826e-04 - val_loss: 2.6586e-04
Epoch 7/20
38/38 [==============================] - 1s 19ms/step - loss: 3.6246e-04 - val_loss: 7.1231e-05
Epoch 8/20
38/38 [==============================] - 1s 18ms/step - loss: 2.9327e-04 - val_loss: 6.0753e-05
Epoch 9/20
38/38 [==============================] - 1s 14ms/step - loss: 2.8803e-04 - val_loss: 3.3790e-05
Epoch 10/20
38/38 [==============================] - 1s 15ms/step - loss: 2.4610e-04 - val_loss: 2.8718e-05
Epoch 11/20
38/38 [==============================] - 1s 16ms/step - loss: 2.7947e-04 - val_loss: 2.3558e-04
Epoch 12/20
38/38 [==============================] - 0s 12ms/step - loss: 2.7165e-04 - val_loss: 3.3310e-05
Epoch 13/20
38/38 [==============================] - 0s 12ms/step - loss: 2.6778e-04 - val_loss: 2.9924e-05
Epoch 14/20
38/38 [==============================] - 0s 13ms/step - loss: 2.7373e-04 - val_loss: 2.6682e-05
Epoch 15/20
38/38 [==============================] - 0s 12ms/step - loss: 2.2202e-04 - val_loss: 1.7306e-04
Epoch 16/20
38/38 [==============================] - 0s 12ms/step - loss: 2.2661e-04 - val_loss: 1.5048e-04
Epoch 17/20
38/38 [==============================] - 0s 12ms/step - loss: 2.4665e-04 - val_loss: 7.3245e-05
Epoch 18/20
38/38 [==============================] - 0s 12ms/step - loss: 2.6511e-04 - val_loss: 3.0512e-04
Epoch 19/20
38/38 [==============================] - 1s 17ms/step - loss: 2.3516e-04 - val_loss: 3.3584e-05
Epoch 20/20
38/38 [==============================] - 1s 23ms/step - loss: 2.3518e-04 - val_loss: 1.6382e-04
In [65]:
# Let's test the functions and get individual stock prices and volumes for AMZN
price_volume_df = individual_stock(stock_price_df, stock_vol_df, 'AMZN')
price_volume_df

# Get the close and volume data as training data (Input)
training_data = price_volume_df.iloc[:, 1:3].values
training_data

# Normalize the data
sc = MinMaxScaler(feature_range = (0, 1))
training_set_scaled = sc.fit_transform(training_data)

# Created the training and testing data, training data contains present day and previous day values
X = []
y = []
for i in range(1, len(price_volume_df)):
    X.append(training_set_scaled [i-1:i, 0])
    y.append(training_set_scaled [i, 0])
    
# Converted the data into array format
X = np.asarray(X)
y = np.asarray(y)

# Splitted the data
split = int(0.7 * len(X))
X_train = X[:split]
y_train = y[:split]
X_test = X[split:]
y_test = y[split:]

# Reshaped the 1D arrays to 3D arrays to feed in the model
X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))
X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))
X_train.shape, X_test.shape

# Created the model
inputs = keras.layers.Input(shape=(X_train.shape[1], X_train.shape[2]))
x = keras.layers.LSTM(150, return_sequences= True)(inputs)
x = keras.layers.Dropout(0.3)(x)
x = keras.layers.LSTM(150, return_sequences=True)(x)
x = keras.layers.Dropout(0.3)(x)
x = keras.layers.LSTM(150)(x)
outputs = keras.layers.Dense(1, activation='linear')(x)

model = keras.Model(inputs=inputs, outputs=outputs)
model.compile(optimizer='adam', loss="mse")
model.summary()

# Trained the model
history = model.fit(
    X_train, y_train,
    epochs = 20,
    batch_size = 32,
    validation_split = 0.2
)

# Made prediction
predicted = model.predict(X)

# Append the predicted values to the list
test_predicted = []

for i in predicted:
  test_predicted.append(i[0])

test_predicted

df_predicted = price_volume_df[1:][['Date']]
df_predicted

df_predicted['predictions'] = test_predicted

df_predicted

# Plot the data
close = []
for i in training_set_scaled:
  close.append(i[0])

df_predicted['Close'] = close[1:]

df_predicted

# Plot the data
interactive_plot(df_predicted, "Original Vs Prediction")
Model: "model_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_3 (InputLayer)         [(None, 1, 1)]            0         
_________________________________________________________________
lstm_6 (LSTM)                (None, 1, 150)            91200     
_________________________________________________________________
dropout_4 (Dropout)          (None, 1, 150)            0         
_________________________________________________________________
lstm_7 (LSTM)                (None, 1, 150)            180600    
_________________________________________________________________
dropout_5 (Dropout)          (None, 1, 150)            0         
_________________________________________________________________
lstm_8 (LSTM)                (None, 150)               180600    
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 151       
=================================================================
Total params: 452,551
Trainable params: 452,551
Non-trainable params: 0
_________________________________________________________________
Epoch 1/20
38/38 [==============================] - 13s 79ms/step - loss: 0.0048 - val_loss: 0.0300
Epoch 2/20
38/38 [==============================] - 0s 13ms/step - loss: 0.0022 - val_loss: 2.0879e-04
Epoch 3/20
38/38 [==============================] - 0s 12ms/step - loss: 1.2856e-04 - val_loss: 4.2253e-05
Epoch 4/20
38/38 [==============================] - 1s 17ms/step - loss: 4.9141e-05 - val_loss: 2.7658e-05
Epoch 5/20
38/38 [==============================] - 1s 21ms/step - loss: 4.6217e-05 - val_loss: 2.4245e-05
Epoch 6/20
38/38 [==============================] - 1s 19ms/step - loss: 3.3276e-05 - val_loss: 2.3238e-05
Epoch 7/20
38/38 [==============================] - 1s 18ms/step - loss: 3.4239e-05 - val_loss: 7.5914e-05
Epoch 8/20
38/38 [==============================] - 1s 23ms/step - loss: 3.4848e-05 - val_loss: 2.1457e-05
Epoch 9/20
38/38 [==============================] - 0s 12ms/step - loss: 3.6663e-05 - val_loss: 3.5832e-05
Epoch 10/20
38/38 [==============================] - 1s 31ms/step - loss: 3.5753e-05 - val_loss: 1.7864e-05
Epoch 11/20
38/38 [==============================] - 0s 13ms/step - loss: 3.3470e-05 - val_loss: 2.6496e-05
Epoch 12/20
38/38 [==============================] - 1s 15ms/step - loss: 3.4934e-05 - val_loss: 2.1422e-05
Epoch 13/20
38/38 [==============================] - 1s 23ms/step - loss: 3.1927e-05 - val_loss: 2.0070e-05
Epoch 14/20
38/38 [==============================] - 1s 23ms/step - loss: 3.3322e-05 - val_loss: 4.5607e-05
Epoch 15/20
38/38 [==============================] - 1s 26ms/step - loss: 3.4408e-05 - val_loss: 1.8041e-05A: 0s - loss: 3.3425
Epoch 16/20
38/38 [==============================] - 0s 13ms/step - loss: 4.0304e-05 - val_loss: 4.5148e-05
Epoch 17/20
38/38 [==============================] - 0s 13ms/step - loss: 4.0194e-05 - val_loss: 6.3983e-05
Epoch 18/20
38/38 [==============================] - 1s 22ms/step - loss: 3.2401e-05 - val_loss: 6.3248e-05
Epoch 19/20
38/38 [==============================] - 1s 26ms/step - loss: 3.4829e-05 - val_loss: 3.9238e-05
Epoch 20/20
38/38 [==============================] - 1s 24ms/step - loss: 4.1328e-05 - val_loss: 2.5701e-05
In [66]:
# Let's test the functions and get individual stock prices and volumes for TSLA
price_volume_df = individual_stock(stock_price_df, stock_vol_df, 'TSLA')
price_volume_df

# Get the close and volume data as training data (Input)
training_data = price_volume_df.iloc[:, 1:3].values
training_data

# Normalize the data
sc = MinMaxScaler(feature_range = (0, 1))
training_set_scaled = sc.fit_transform(training_data)

# Created the training and testing data, training data contains present day and previous day values
X = []
y = []
for i in range(1, len(price_volume_df)):
    X.append(training_set_scaled [i-1:i, 0])
    y.append(training_set_scaled [i, 0])
    
# Converted the data into array format
X = np.asarray(X)
y = np.asarray(y)

# Splitted the data
split = int(0.7 * len(X))
X_train = X[:split]
y_train = y[:split]
X_test = X[split:]
y_test = y[split:]

# Reshaped the 1D arrays to 3D arrays to feed in the model
X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))
X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))
X_train.shape, X_test.shape

# Created the model
inputs = keras.layers.Input(shape=(X_train.shape[1], X_train.shape[2]))
x = keras.layers.LSTM(150, return_sequences= True)(inputs)
x = keras.layers.Dropout(0.3)(x)
x = keras.layers.LSTM(150, return_sequences=True)(x)
x = keras.layers.Dropout(0.3)(x)
x = keras.layers.LSTM(150)(x)
outputs = keras.layers.Dense(1, activation='linear')(x)

model = keras.Model(inputs=inputs, outputs=outputs)
model.compile(optimizer='adam', loss="mse")
model.summary()

# Trained the model
history = model.fit(
    X_train, y_train,
    epochs = 20,
    batch_size = 32,
    validation_split = 0.2
)

# Made prediction
predicted = model.predict(X)

# Append the predicted values to the list
test_predicted = []

for i in predicted:
  test_predicted.append(i[0])

test_predicted

df_predicted = price_volume_df[1:][['Date']]
df_predicted

df_predicted['predictions'] = test_predicted

df_predicted

# Plot the data
close = []
for i in training_set_scaled:
  close.append(i[0])

df_predicted['Close'] = close[1:]

df_predicted

# Plot the data
interactive_plot(df_predicted, "Original Vs Prediction")
Model: "model_3"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_4 (InputLayer)         [(None, 1, 1)]            0         
_________________________________________________________________
lstm_9 (LSTM)                (None, 1, 150)            91200     
_________________________________________________________________
dropout_6 (Dropout)          (None, 1, 150)            0         
_________________________________________________________________
lstm_10 (LSTM)               (None, 1, 150)            180600    
_________________________________________________________________
dropout_7 (Dropout)          (None, 1, 150)            0         
_________________________________________________________________
lstm_11 (LSTM)               (None, 150)               180600    
_________________________________________________________________
dense_3 (Dense)              (None, 1)                 151       
=================================================================
Total params: 452,551
Trainable params: 452,551
Non-trainable params: 0
_________________________________________________________________
Epoch 1/20
38/38 [==============================] - 11s 71ms/step - loss: 0.0054 - val_loss: 0.0080
Epoch 2/20
38/38 [==============================] - 1s 23ms/step - loss: 0.0021 - val_loss: 4.4243e-04
Epoch 3/20
38/38 [==============================] - 1s 19ms/step - loss: 1.1797e-04 - val_loss: 6.9372e-05
Epoch 4/20
38/38 [==============================] - 1s 23ms/step - loss: 4.6179e-05 - val_loss: 5.1779e-05
Epoch 5/20
38/38 [==============================] - 1s 18ms/step - loss: 4.5080e-05 - val_loss: 1.9167e-05
Epoch 6/20
38/38 [==============================] - 0s 13ms/step - loss: 3.8583e-05 - val_loss: 1.7503e-05
Epoch 7/20
38/38 [==============================] - 1s 18ms/step - loss: 3.5152e-05 - val_loss: 2.2150e-05
Epoch 8/20
38/38 [==============================] - 1s 20ms/step - loss: 3.4338e-05 - val_loss: 1.9641e-05
Epoch 9/20
38/38 [==============================] - 1s 21ms/step - loss: 3.5033e-05 - val_loss: 1.7696e-05
Epoch 10/20
38/38 [==============================] - 1s 15ms/step - loss: 3.4460e-05 - val_loss: 2.3175e-05
Epoch 11/20
38/38 [==============================] - 1s 17ms/step - loss: 3.4602e-05 - val_loss: 1.7085e-05
Epoch 12/20
38/38 [==============================] - 1s 16ms/step - loss: 3.4874e-05 - val_loss: 2.0532e-05
Epoch 13/20
38/38 [==============================] - 1s 23ms/step - loss: 3.1974e-05 - val_loss: 2.7085e-05
Epoch 14/20
38/38 [==============================] - 1s 13ms/step - loss: 3.9748e-05 - val_loss: 4.9369e-05
Epoch 15/20
38/38 [==============================] - 1s 26ms/step - loss: 3.4446e-05 - val_loss: 1.7301e-05
Epoch 16/20
38/38 [==============================] - 1s 26ms/step - loss: 3.7685e-05 - val_loss: 1.8711e-05
Epoch 17/20
38/38 [==============================] - 1s 27ms/step - loss: 3.7481e-05 - val_loss: 1.7067e-05
Epoch 18/20
38/38 [==============================] - 1s 27ms/step - loss: 3.1405e-05 - val_loss: 3.7116e-05
Epoch 19/20
38/38 [==============================] - 1s 28ms/step - loss: 3.2013e-05 - val_loss: 1.8680e-05
Epoch 20/20
38/38 [==============================] - 1s 13ms/step - loss: 3.2370e-05 - val_loss: 2.6979e-05
In [67]:
# Let's test the functions and get individual stock prices and volumes for AAPL
price_volume_df = individual_stock(stock_price_df, stock_vol_df, 'AAPL')
price_volume_df

# Get the close and volume data as training data (Input)
training_data = price_volume_df.iloc[:, 1:3].values
training_data

# Normalize the data
sc = MinMaxScaler(feature_range = (0, 1))
training_set_scaled = sc.fit_transform(training_data)

# Created the training and testing data, training data contains present day and previous day values
X = []
y = []
for i in range(1, len(price_volume_df)):
    X.append(training_set_scaled [i-1:i, 0])
    y.append(training_set_scaled [i, 0])
    
# Converted the data into array format
X = np.asarray(X)
y = np.asarray(y)

# Splitted the data
split = int(0.7 * len(X))
X_train = X[:split]
y_train = y[:split]
X_test = X[split:]
y_test = y[split:]

# Reshaped the 1D arrays to 3D arrays to feed in the model
X_train = np.reshape(X_train, (X_train.shape[0], X_train.shape[1], 1))
X_test = np.reshape(X_test, (X_test.shape[0], X_test.shape[1], 1))
X_train.shape, X_test.shape

# Created the model
inputs = keras.layers.Input(shape=(X_train.shape[1], X_train.shape[2]))
x = keras.layers.LSTM(150, return_sequences= True)(inputs)
x = keras.layers.Dropout(0.3)(x)
x = keras.layers.LSTM(150, return_sequences=True)(x)
x = keras.layers.Dropout(0.3)(x)
x = keras.layers.LSTM(150)(x)
outputs = keras.layers.Dense(1, activation='linear')(x)

model = keras.Model(inputs=inputs, outputs=outputs)
model.compile(optimizer='adam', loss="mse")
model.summary()

# Trained the model
history = model.fit(
    X_train, y_train,
    epochs = 20,
    batch_size = 32,
    validation_split = 0.2
)

# Made prediction
predicted = model.predict(X)

# Append the predicted values to the list
test_predicted = []

for i in predicted:
  test_predicted.append(i[0])

test_predicted

df_predicted = price_volume_df[1:][['Date']]
df_predicted

df_predicted['predictions'] = test_predicted

df_predicted

# Plot the data
close = []
for i in training_set_scaled:
  close.append(i[0])

df_predicted['Close'] = close[1:]

df_predicted

# Plot the data
interactive_plot(df_predicted, "Original Vs Prediction")
Model: "model_4"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_5 (InputLayer)         [(None, 1, 1)]            0         
_________________________________________________________________
lstm_12 (LSTM)               (None, 1, 150)            91200     
_________________________________________________________________
dropout_8 (Dropout)          (None, 1, 150)            0         
_________________________________________________________________
lstm_13 (LSTM)               (None, 1, 150)            180600    
_________________________________________________________________
dropout_9 (Dropout)          (None, 1, 150)            0         
_________________________________________________________________
lstm_14 (LSTM)               (None, 150)               180600    
_________________________________________________________________
dense_4 (Dense)              (None, 1)                 151       
=================================================================
Total params: 452,551
Trainable params: 452,551
Non-trainable params: 0
_________________________________________________________________
Epoch 1/20
38/38 [==============================] - 12s 87ms/step - loss: 0.0059 - val_loss: 0.0209
Epoch 2/20
38/38 [==============================] - 1s 24ms/step - loss: 0.0022 - val_loss: 0.0074
Epoch 3/20
38/38 [==============================] - 1s 24ms/step - loss: 5.8246e-04 - val_loss: 1.6374e-05
Epoch 4/20
38/38 [==============================] - 1s 25ms/step - loss: 5.7841e-05 - val_loss: 1.6759e-05
Epoch 5/20
38/38 [==============================] - 1s 15ms/step - loss: 5.0340e-05 - val_loss: 1.7435e-05
Epoch 6/20
38/38 [==============================] - 1s 15ms/step - loss: 4.1142e-05 - val_loss: 2.0632e-05
Epoch 7/20
38/38 [==============================] - 1s 17ms/step - loss: 3.8122e-05 - val_loss: 2.0390e-05
Epoch 8/20
38/38 [==============================] - 1s 20ms/step - loss: 3.9513e-05 - val_loss: 1.6314e-05
Epoch 9/20
38/38 [==============================] - 1s 14ms/step - loss: 3.9367e-05 - val_loss: 2.6303e-05
Epoch 10/20
38/38 [==============================] - 1s 16ms/step - loss: 4.0847e-05 - val_loss: 1.6980e-04
Epoch 11/20
38/38 [==============================] - 1s 20ms/step - loss: 4.1846e-05 - val_loss: 2.2310e-05
Epoch 12/20
38/38 [==============================] - 1s 14ms/step - loss: 3.9788e-05 - val_loss: 1.7754e-05
Epoch 13/20
38/38 [==============================] - 1s 24ms/step - loss: 4.1384e-05 - val_loss: 3.6255e-05
Epoch 14/20
38/38 [==============================] - 1s 22ms/step - loss: 3.6499e-05 - val_loss: 2.3386e-05
Epoch 15/20
38/38 [==============================] - 1s 19ms/step - loss: 3.4176e-05 - val_loss: 2.9690e-05
Epoch 16/20
38/38 [==============================] - 1s 20ms/step - loss: 3.7671e-05 - val_loss: 2.3578e-05
Epoch 17/20
38/38 [==============================] - 1s 20ms/step - loss: 4.0845e-05 - val_loss: 2.8944e-05
Epoch 18/20
38/38 [==============================] - 1s 22ms/step - loss: 3.5473e-05 - val_loss: 1.7415e-05
Epoch 19/20
38/38 [==============================] - 1s 14ms/step - loss: 3.6206e-05 - val_loss: 1.1970e-04
Epoch 20/20
38/38 [==============================] - 1s 14ms/step - loss: 4.1910e-05 - val_loss: 6.2152e-05